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Abstract  of  Dissertation  Presented  to  the 

Graduate  Council  of  the  University  of  Florida  in  Partial  Fulfillment 

of  the  Requirements  for  the  Degree  of  Doctor  of  Philosophy 

ON  SOIVEE  PROBLEMS  OF  ESTIMATION  AND  PREDICTION 
FOR  NON- STATIONARY  TIME  SERIES 

By 

James  Thomas  McClave 
March,  1971 

Chairman:   Dr.  Kamal  C.  Chanda 
Major  Department:   Statistics 

Many  techniques  are  available  for  estimating  the  parameters  of 
linear  stationary  time  series.   The  effect  of  some  of  these  estimators 
on  the  least  squares  predictor  for  some  future  value  of  the  series  is 
examined.   We  have  obtained  approximations  for  the  increase  in  predic- 
tion error  due  to  parameter  estimation  in  several  cases  for  which  no 
exact  expression  could  be  found. 

An  efficient  estimator  for  the  parameters  in  a  first  order 
autoregressive-moving  average  model  is  developed  by  making  use  of  a 
linear  function  of  the  autocorrelations.   For  more  general  models  we 
conclude  that  efficient  estimation  is  so  difficult  to  attain  that  first 
consideration  in  many  estimation  prediction  problems  should  be  given 
to  ease  of  calculation. 

Numerous  unsolved  estimation  and  prediction  problems  remain 
for  non-stationary  time  series.   We  consider  the  logistic  growth  proc- 
ess, which  is  used  extensively  as  an  economic  and  population  growth 
model,  in  detail.   Current  estimation  procedures  for  the  logistic 
process'  parameters  make  no  reference  to  an  error  structure. 


We  propose  a  probability  structure  consistent  with  the  realistic 
properties  of  the  series.   We  then  use  this  structure  to  obtain  esti- 
mators from  three  different  observational  standpoints: 

(1)  that  of  observation  at  equidistant  time  points, 

(2)  that  of  continuous  observation,  and 

(3)  that  of  arrival  time  observation. 

For  observation  types  (1)  and  (2)  we  have  used  a  modification 
of  maximum  likelihood  procedures  to  obtain  estimators  having  most  of 
the  usual  properties  associated  with  maximum  likelihood  estimators. 
A  computer  program  was  written  for  type  (1)  to  solve  the  intractable 
estimation  equations,  using  the  Newton-Rhapson  procedure.   Observation 
of  arrival  times  is  shown  not  to  lead  to  any  useful  estimation  proce- 
dures.  We  also  examine  the  effect  of  the  estimators  calculated  from 
the  observations  taken  at  equal  time  intervals  (type  (1)  above)  on  the 
error  of  prediction. 

The  procedure  developed  for  observation  type  (1)  is  then  compared 
by  means  of  example  to  an  estimation  procedure  developed  by  Rhodes. 
The  logistic  model  is  also  fitted  by  each  method  to  the  population  of 
conterminous  United  States.   The  estimates  are  then  used  to  obtain 
predictions  of  the  population  of  conterminous  United  States  in  1970. 
We  conclude  from  the  results  that  the  effort  required  to  calculate  the 
maximum  likelihood  estimates  is  worthwhile.   We  further  conjecture  that 
the  methods  developed  may  be  applicable  to  other  growth  processes. 


CHAPTER  I 


STATEMEJfT  OF  THE  PROBLEM 


1. 1   Introduction 

Consider  the  following  two  types  of  time  series: 

1.  fx(t) ;  t  e  J   ,  ],  where  J   ,  is  the  set  of  real  integers 

a.b-*         a,b 

{a,a-!-l,...  ,b}.   This  is  the  discrete  time  series. 

2.  {X(t) ;  t  e  I   ,  ],  where  I   ,  is  the  interval  (open  or  closed) 

a,b^         a ,b 

between  the  real  numbers  a  and  b.   This  is  the  continuous 

time  series. 
Suppose  an  observation  is  made  on  a  given  series,  either  obtain- 
ing an  observation  set  of  the  kind  [x(S),x(S+l) x(T) ] ,  with  S  and 

T  real  integers  such  that  J    CI  j     from  the  first  or  second  type; 
or  of  the  kind  [xCt) ;  t  €  I    },  with  I    c  I   .  ,  from  the  second  type. 

Of!  ^  t  -^      3.  J  t) 

The  problem  to  be  considered  is  that  of  finding  the  least  squares  pre- 
dictor of  X(T-hm)  (with  m  an  integer  if  the  series  is  discrete  and  T+m  s  b) 
based  upon  the  observation.   Thus,  we  want  to  find  that  function  of  the 
observation  set,  say  x  (m) ,  which  minimizes  the  mean  square  error  of 
prediction. 

The  first  step  toward  the  solution  of  this  problem  is  to  construct 
a  parametric  representation  of  the  series.   One  must  make  use  of  all  avail- 
able information  .ibout  [X(t)  ]  in  order  to  place  the  problem  in  a  parametric 
framework.   This  parametrization  may  vary  from  specification  of  the  first 


and/or  second  moments  of  the  series  to  complete  specification  of  the 
probability  distribution  at  each  point  t. 

This  study  of  the  series'  genesis  will  usually  place  the  process 
into  one  of  two  subclasses: 

(a)  Weakly  stationary,  hereafter  referred  to  as  stationary, 
series;  that  is  to  say,  those  whose  first  two  moments  are, 
for  every  t , 

E{X(t)]=  ^L,  (1.1.1) 

independent  of  t ;  and 

E{X(t+s)X(t)}  =  Y  .  (1.1.2) 

a  function  of  s  only. 

(b)  Non-stationary  series,  the  complement  of  subclass  (a). 
If  the  series  satisfies  the  conditions  of  stationarity ,  the 

prediction  problem  is  simplified.   The  minimum  mean  square  error  pre- 
dictor depends  on  the  first  two  moments  of  the  time  series.   Since  for 
stationary  processes  the  second  moment  depends  only  upon  the  "lag"  or 
time  difference  of  the  random  variables,  and,  since  the  first  moment  is 
independent  of  time,  all  sequences  of  observations  which  are  of  equal 
lengths  yield  the  same  information  about  the  series'  first  two  moments, 
no  matter  when  in  the  series'  development  the  observations  are  taken. 
This  second  order  homogeneity  simplifies  the  problem  to  the  extent  that 
all  further  parametrization  pertains  to  the  description  of  ^  and  y    , 
from  which  the  predictor  will  be  calculated. 

On  the  other  hand,  if  the  series  is  non-stationary,  the  problem 
is  more  complex  because  of  the  nature  of  this  subclass.  We  will  divide 
the  non-stationary  subclass  into  two  categories: 


(i)  those  for  which  a  simple  transformation  exists  which  trans- 
forms the  process  into  a  stationary  time  series,  and 
(ii)  those  for  which  either  no  such  transformation  exists,  or  at 

least  no  simple  transformation  can  be  found. 
As  to  the  first  category,  the  nature  of  the  transformation 
depends,  of  course,  on  the  reason  for  the  series'  non-stationarity. 
As  a  simple  example,  suppose  {X(t)]  is  a  continuous  time  process  with 

E{X(t)3  -  ^(t)  =  a  +  0t,  (1.1.3) 

E[X(t)  X(t+s)}  =  Yg.  (1.1.4) 

Then  the  process  [Y(t)],  formed  by  taking 

Y(t)  =  X(t)  -  m(t),  (1.1.5) 

will  be  stationary. 

This  particular  kind  of  transformation  is  popularly  referred  to 
as  removal  of  the  trend  from  [X(t)];it  can  be  accomplished  for  simple 
linear  trends  like  that  given  by  (1.1.3),  or  for  more  complex  seasonal 
or  harmonic  trends  common  to  many  economic  time  series.   The  end  result 
is  the  same;  this  further  parametrization  has  permitted  us  to  reduce  the 
problem  to  that  of  stationary  series  prediction,  for  which  solutions 
are  known. 

However,  it  is  perhaps  within  the  second  category  that  one 
encounters  the  most  complex  prediction  problems  in  time  series.   Without 
the  homogeneity  of  the  first  two  moments,  the  non-stationary  process' 
analysis  will  certainly  depend  on  when  the  observation  is  taken;  and  the 
underlying  probability  structure  of  the  series  can  be  extremely  compli- 
cated.  In  short,  general  methods  cannot  be  used  to  solve  the  prediction 
problem  for  non-stationary  processes  with  satisfactory  results.   Each 


series  must  be  considered  individually,  its  evolution  thoroughly  re- 
searched, a  parametrization  made,  and  then  an  optimal  prediction  proce- 
dure developed.   One  can  hope  that  this  systematic  method  will  apply  to 
a  general  group  of  processes,  but  that  the  actual  procedure  developed 
will  apply  to  any  but  the  specific  series  being  investigated  is  extremely 
doubtful. 

1.2  History — Stationary  Processes 

The  literature  on  the  prediction  problem  for  the  stationary  time 
series  is  voluminous.   That  the  area  is  extremely  attractive  follows  from 
the  above;  most  results  have  wide  application  within  the  subclass  of 
stationary  processes,  and  the  chances  that  a  problem  is  mathematically 
tractable  are  enhanced  by  stationarity.   In  addition,  of  course,  station- 
ary processes  have  great  practical  significance. 

Problems  of  least  squares  prediction  for  stochastic  processes 
were  first  considered  by  Kolmogorov  [20] ,  using  Hilbert  space  geometry. 
This  treatment  has  been  adapted  to  the  stationary  processes  by  several 
authors,  including  Doob  [9],  Yaglom  [36],  and  Parzen  [24].   Hence  the 
prediction  problem  is  essentially  solved  for  all  stationary  time  series; 
the  known  general  methods  need  only  be  applied  to  the  specific  stationary 
process  being  considered.   Many  authors  have  contributed  to  this  end, 
among  them  Doob  [9],  Yaglom  [36],  Whittle  [33,34],  Box  and  Jenkins  [3,4], 
and  Jenkins  and  Watts  [17]. 

The  particular  parametrization  proposed  often  results  in  an 
estimation  problem  which  must  be  solved  prior  to  making  the  prediction. 
This  is  illustrated  by  three  of  the  most  commonly  hypothesized  dis- 
crete time  series:   the  autoregressive,  moving  average,  and  mixed 


autoregressive-moving  average  (hybrid)  models.   Each  receives  wide  appli- 
cation in  engineering  and  economic  research,  and  each  entails  parameter 
estimation.   These  estimation  problems  have  been  considered  by  Mann  and 
Wald  [21],  Wliittle  [33],  IXirbin  [10,11,12],  Walker  [28,29,30,31], 
Hannan  [16],  and  Box  and  Jenkins  [4].   Some  of  the  estimation  procedures 
are  rather  complicated  to  apply;  but,  for  the  most  part,  satisfactory 
estimation  methods  exist  for  these  linear  stationary  models. 

The  literature  is  nearly  devoid  of  research  exiDloring  the  effect 
of  parameter  estimation  on  the  error  of  least  square  predictors.   Even 
though  the  error  increase  due  to  parameter  estimation  will  usually  be  of 
a  smaller  order  than  the  mean  square  error,  the  precise  nature  of  this 
increase  needs  to  be  studied.   Box  and  Jenkins  [4]  do  present  a  brief 
discussion  of  this  problem  for  one  simple  time  series  model,  but  to  my 
knowledge  the  subject  has  not  been  mentioned  elsewhere.   This  problem 
will  be  considered  at  some  length  in  Chapter  II. 

1-3  History — Non-stationary  Processes 

All  the  work  on  the  prediction  problem  for  non-stationary  time 
series  is  divided  into  two  distinct  groups:   that  which  deals  with  trend 
removal  in  order  to  make  the  process  stationary,  and  that  which  does  not. 
We  shall  consider  only  the  latter  in  this  dissertation;  the  interested 
reader  may  consult  Grenander  and  Rosenblatt  [15]  and  Box  and  Jenkins  [4] 
for  a  complete  treatment  of  trend  removal. 

Kolmogorov's  Hilbert  space  solution  to  the  prediction  problem 
is  applicable  to  non-stationary  processes.   The  solution  depends  mainly 
on  the  second  oi-dor  properties  of  the  process.   Since  the  second  moments 


are  inherently  nore  complex  for  non-stationary  processes  than  those  for 
stationary  processes,  the  result  of  the  application  of  Kolmogorov' s 
method  to  a  non-stationary  time  series  usually  does  not  have  general 
application  within  the  class  of  non-stationary  processes.   For  this 
reason  one  finds  that  most  of  the  estimation  and  prediction  research  in 
this  area  has  dealt  with  specific  process  types;  for  example,  queuing 
processes,  renewal  processes,  birth-death  processes,  and  growth 
(evolutive)  processes.   The  list  is  not  exhaustive;  each  class  has 
received  attention  in  the  literature,  much  of  it  devoted  to  the  estima- 
tion and  prediction  problems. 

Specifically,  considering  the  birth-death  and  growth  processes, 
t>oth  usually  Markov  processes,  most  of  the  research  has  been  concen- 
trated on  homogeneous  processes;  that  is,  those  with  transition  probabil- 
ities independent  of  time.   Feller  [13]  gives  a  rather  complete  intro- 
duction to  the  birth-death  processes,  defining  the  underlying  probabil- 
ity structure  of  them.   Then  David  Kendall  [18]  uses  generating  functions 
to  obtain  solutions  for  the  probability  distribution  of  these  series  at 
any  time  t.   Moran  [23]  partly  solves  the  problem  of  estimation  for 
homogeneous  birth-death  processes,  and  both  Moran  [22]  and  Kendall  [19] 
propose  several  r.iethods  for  estimating  the  parameters  of  the  transition 
probabilities  of  some  simple  non-homogeneous  birth-death  and  growth 
processes. 

Since  the  publication  of  these  papers,  research  in  this  area  has 
been  sparse  at  best;  and  the  prediction  problem  has  received  surpris- 
ingly little  attention  from  statisticians.   Biologists  and  economists 


(Rhodes  [25],  Tinter  [26],  and  Granger  [14])  have  taken  some  interest  in 
this  area,  since  it  includes  prediction  of  animal  population  size  and 
economic  forecasting.   Unfortunately,  these  problems  are  often  treated 
with  little  reference  to  underlying  probability  structures.   The  results 
rely  more  on  intuition  than  mathematics,  and  hence  the  predictions  are 
often  unsatisfactory. 

An  example  of  this  kind  of  treatment  is  given  by  Rhodes  [25]. 
Rhodes  considers  the  problem  of  estimating  human  population  size  without 
using  any  probabilistic  arguments.   His  early  paper  was  an  indication  of 
things  to  come.   In  all  fairness  though,  early  researchers  had  a  legit- 
imate excuse  for  steering  away  from  these  arguments,  for  they  often  make 
prediction  problems  for  non-homogeneous  Markov  processes  mathematically 
intractable.   However,  in  this  age  of  the  electronic  computer,  many 
formerly  insoluble  problems  can  be  treated  iteratively  and  solutions 
obtained. 

1.4   Summary  of  Results 

The  results  are  divided  into  two  distinct  categories:  first,  in 
Chapter  II  the  effect  of  estimation  on  the  error  of  prediction  for  linear 
stationary  time  series  is  considered.   For  several  models  an  exact  expres- 
sion for  the  error  increase  due  to  parameter  estimation  is  obtained;  in 
many  cases  for  which  this  is  not  possible,  approximations  for  this 
increase  are  given.   For  the  mixed  autoregressive-moving  average  model, 
a  new  estimation  procedure  is  examined  with  respect  to  the  prediction 
problem. 


In  Chapter  III  the  logistic  growth  process,  a  non-stationary, 
Markov  time  series,  is  considered  in  detail.   A  realistic  probabilistic 
structure  is  assigned  to  this  process ,  and  then  the  parameters  involved 
are  estimated  from  several  different  observational  standpoints.   Finally, 
the  prediction  problem  is  considered  for  this  process,  and  the  effect 
of  parameter  estimation  on  the  mean  square  error  of  prediction  is 
examined. 

The  methods  developed  for  the  logistic  process  may  be  applicable 
to  other  non-stationary  processes.   Nevertheless,  the  logistic  process 
has  application  in  describing  animal  population  growth  and  certain  eco- 
nomic growth  patterns,  so  that  the  results  may  be  useful  in  themselves. 

Chapter  IV  is  devoted  to  a  comparison  of  an  estimation  procedure 
(for  the  logistic  process)  derived  by  Rhodes  [25]  with  one  of  those 
derived  in  this  paper.   The  analytical  intractability  of  the  estimators' 
sampling  properties  calls  for  numerical  comparison,  so  that  conclusions 
are  tentative.   The  purpose  of  this  Monte  Carlo  study  is  to  determine 
whether  the  extra  effort  which  the  probabilistic  methods  require  is 
worthwhile,  or  whether  one  should  continue  to  estimate  the  parameters, 
using  methods  like  those  proposed  by  Rhodes;  that  is  to  say,  those  which 
rely  mainly  on  intuitive  appeal  and  their  ease  of  calculation. 


CHAPTER  II 


LINEAR  STATIONARY  MODELS 


2. 1  Introduction 

Many  volumes  have  been  written  about  estimation  and  prediction 
problems  for  linear  stationary  time  series,  but  there  is  little  in  the 
literature  which  considers  the  effect  of  estimation  upon  prediction. 
That  is,  the  increase  in  the  mean  square  error  of  prediction  due  to 
parameter  estimation  has  not  been  examined. 

Knowledge  about  the  increase  in  prediction  error  for  various 
predictors  may  be  useful.   If  the  increase  is  small  for  several  differ- 
ent estimators,  one  might  choose  the  estimator  easiest  to  calculate. 
On  the  other  hand,  one  would  spare  no  effort  in  finding  that  estimator 
which  least  increased  the  error  of  prediction  if  that  increase  were 
known  to  be  significant. 

Our  purpose  in  this  chapter  is  to  consider  this  estimation- 
prediction  problem  for  the  three  basic  linear  stationary  time  series: 
autoregressive,  moving  average,  and  a  mixture  of  these  (hybrid).   Only 
discrete  time  series  will  be  examined;  since  the  stationarity  of  the 
process  implies  that  the  mean  is  mathematically  independent  of  time, 
this  mean  will  be  assumed  known  and  equal  to  zero.   All  results  remain 
asymptotically  true  for  the  case  of  an  unknown  mean  for  the  series 
formed  by  subtracting  the  arithmetic  mean  from  each  of  the  observed 
values.   To  pursue  this  would  be  stray  from  the  objective;  the  inter- 
ested reader  is  referred  to  Bartlett  [1]  and  Box  and  Jenkins  [4]. 

9 


10 


Finally,  the  sequence  [Z  ;  t  =  0,  ±1,...}  will  be  used  throughout 
to  denote  a  stationary  series  whose  first  two  moments  are  the  following: 
E[Z^}  =  0,  (2.1.1) 


rcl  if   j  =  0 

^   ^^J     ^0      if   j  p^  0  . 


From  time  to  time  comment  will  be  made  on  the  case  where  Z   is  normally 
distributed.   But  unless  specified,  no  distributional  assumptions  are 
made. 

2. 2   The  Autoregressive  Sequence 

If  fX  :  t  e  J     1  is  a  stationary  time  series  satisfying  the 

I.   •(;  '  _00  00-' 

relationship 

K  +   a,X^  ,  +  ...+  Q-  X^   =  Z.  (2.2.1) 

t    1  t-1  p  t-p    t 

for  every  t,then  [X  }  is  said  to  be  a  p   order  autoregressive  sequence. 

The  problem  of  estimating  the  set  of  parameters  [a.;    j    ^  ^i  -^ 

is  analogous  to  the  classical  regression  problem.   Defining  y.    as  the 

j-step  covariance  of  [X  }  and  p  .  as  the  j-step  correlation,  that  is 
t       J 

to  say 


Y.  =  EfX^  .  X^}  (2.2.2) 

'J      t+j   t-" 


and 


we  get 


-1 
'j  -   Yj^  • 


P •  -  YYr  •  (2.2.3) 


E  Oi.y.        =   Y 
j=l  ^  ^-"    " 


(2.2.4) 


for  r  e  J    by  multiplying  (2.2.1)  by  X    and  taking  expected  values. 

We  use  the  fact  that  Z   and  X    are  uncorrelated  when  r  <  0,  which 
t      t-r 
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will  be  sho-.m  later.   Equation  (2.2.4)  gives  rise  to  the  estimating 

equations 

P 

Z  a  C    =  C  (2.2.5) 

_   i   i-r    r 

for  r  e  J,   ,  where 
1,P 

is  the  sar.ple  covariance  function  for  the  observed  sequence 

{x  ;  t  e  J    }.   Mann  and  Wald  [21]  have  shown  that  the  estimators 

fa  ;  J  e  J    ]  given  by  (2.2.5)  are  asymptotically  normally  distributed 
"•  j       l.P 

-1   2   -1 
with  null  mean  and  variance  covariance  matrix   (^Yq)    ^  Z   ,  where 

E  is  a  (p  xp)  matrix  whose  element  in  the  (i,j)  cell  is 

(D.  .  =  0.  (2.2.7) 

The  vector  of  estinates  whose  transpose  is  q^  =  (a^  ,Q!^,  ■  ■  ■  ,0   )  is 

T 
shown  to  be  the  maximum  likelihood  estimator  of  a  =  (a^ ,a^,  ■  .  .  ,a  )    if 

the  distribution  of  Z   is  assumed  normal  for  each  t. 

In  order  to  predict  the  value  of  X    using  the  observations 

X  ,x  , ... ,x.,  we  will  use  the  function  which  minimizes  the  mean  square 

error  of  prediction.   That  is,  the  optimal  predictor  is  x^   (m)  which 


minimizes 


MSE[^f/\m)}  =  Ef[X^^   -}^f/\m)]^}  .  (2.2.8) 

"■  N     -'     ^   N+m   N       ■' 


But  this  irolies  that 


^.^\m)  =  EX.  /(X.  =  X.;  j  e  J^   )}  .  (2.2.9) 

>  -  N+m   J     J        1,N  ■' 
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Substitution  of  (2.2.1)  into  (2.2.9)  yields 


xA  '(m)  =.  E{[Z   -  Z     a.  X         .]/(X.  =  x.;j  e  J,  ^)} 
N  N+m   ._   J   N+m-j     j    j  ""     1,N  "' 


=   -     T.     a.    X  (m-j) 
J=l  J   ^ 


(2.2.10) 


where 


^»a)  = 


E{X   ./(X  =x  •  k  e  J,   )] 
"•  N+j    k    k       1,N  -^ 

if  j  >  0 

'Sj.^  if  j   ^0 


(2.2.11) 


'J+J 


Thus  one  can  calculate  the  m-step  predictor,  x„(in)  ,  using  the  recursive 

N 

relation  (2.2.10). 

This,  however,  requires  that  the  autoregressive  parameters  be 
known.   When  they  are  not,  we  may  logically  use 


(2) 


^(2) 


3C.   (m)  =  -  E  a.    -x-^        (m-j) 


(2.2.12) 


To  be  determined  is  the  increase  in  the  error  of  prediction  caused  by 

using  X    (m)  instead  of  x    (m)  . 

N  N 


Case  1:   p  =  m  =  1 

For  the  sake  of  simplicity,  let  a  =   -  a^  ,    so  that  fX  ]  satisfies 


the  relationship 


^t  -  ^\-l   =  \ 


for  all  t.   Using  the  Mann-Wald  estimate  for  a,    one  has 
-1 


and 


a  .  c^  c^ 

Var  (a)  «  ^^Yq^'^   ^-^ 


(2.2.13) 

(2.2.14) 
(2.2.15) 
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Rewriting  (2.2.13)  and  using  the  fact  that  the  stationarity  of  [X  } 


implies  \a\    <  1,  one  obtains 


=  0-  X^  .  +   E  c?    Z^    .  (2.2.16) 

for  any  positive  integer  k.   We  can  extend  this  to  the  limit  in  the 
mean  square  sense  as  k  -•  =.   That  is,  one  can  write 

00 

X.  =  E  a^   1^    .,  (2.2.17) 

'   j=0    "-' 


*    J=0      2 

=  (l-a^)~^  a^  (2.2.18) 

is  finite.   Note  that  the  left  side  of  (2.2.18)  is,  by  definition,  y^ ' 
and 


=  4[A"'^..<-j]lI«' vj} 


-j-0      ^-^  -j=o 


=  (^   Y^.  (2.2.19) 


Thus 


Pk  "  Yk  Yo   =  »   •  (2.2.20) 

In  (2.2.19)  and  (2.2.20)  we  assume  k  is  a  positive  integer;  that  Y  t  ~  Yi, 
is  easily  shown. 
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The  minimum  mean  square  error  predictor  of  X    is  given  by- 
setting  p  =  m  =  1  in  (2.2.10),  whereupon 

^^\l)    =  ax^.  (2.2.21) 

The  prediction  error  is  given  by 

=  ^f<\.i  -  ^v'^ 

=  a^.  (2.2.22) 

If  a   is  unknown,  the  predictor  corresponding  to  (2.2.12)  is 
x^^\l)  -   ax^.  (2.2.23) 

Using  the  fact  that  [X    -  X    (1)]  is  uncorrelated  with  the  sequence 

N+1     N 


{X  ;  j  e  J^  ^3,  one  has 


=  MSE,  +  E[xl(a-a)^].  (2.2.24) 

IN 

Thus  the  increase  in  the  error  of  prediction  due  to  estimating  a   is 

MSE„  -  MSE,  =  Efxffa-Q')^]  .  (2.2.25) 

2      IN       -' 

Suppose  the  estimator  a   is  calculated  using  only  the  first  k 

observations  in  the  sequence  [x.;  j  €  J,  „},  where  k  «  N;  that  is  to  say, 

we  form  the  sample  covariances  in  (2.2.3),  using  only  k  observations. 

Since  p.  decreases  exponentially,  the  [X . ;  j  e  J    }  and  X   are  virtually 

uncorrelated,  and  hence  so  are  a   and  X^.   A  first  approximation  for 

(2.2.25)  is  thus 
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=  Yq  k"^(l-a^)  =  k"^  a^    .  (2.2.26) 

Box  and  Jenkins  [4]  present  an  argument  similar  to  the  above.   The 
degree  to  which  the  approximation  (2.2.24)  holds  is  impossible  to  ascer- 
tain from  these  £j-guments. 

To  obtain  a  more  exact  expression  for  (2.2.25),  first  define 

6   =  C   -  V  (2.2.27) 

J    J    M 

for  j  e  J    .   Then  we  can  write 
l,h 

(a-ci)  =  C^  C~  -a 

=  (5„  +  Yn)'^  (S^  -  cfi^),  (2.2.28) 


^0  ^  Yo^    ^"i  -  ^0^ 


using  the  fact  that  y.,  ~  '^Yn  ~  ^• 


Then 


Ef(5-a)-}  =  E{(6^-c.6^)2  (6q  +  Yo)'^} 

=  E[y-'(6^-P  6^)2  (l-26^Yo'- 3(6^  Yo')'-  •••)3 

=  EfYQ^(6^-p  6^)2}  +  o(n"^),  (2.2.29) 

—1/2 
since  £   and  i      have  sampling  errors  whose  order  of  magnitude  is  N 

(Bartlett  [1]).   Thus,  retaining  terms  of  order  N   and  higher,  (2.2.25) 


becomes 


E{xJ(^:-c.)2}  «Yo^  ^f^^^-^V^^  ^2.2.30) 


We  now  state 


Theorem  2.2.1: 


If  ^X  ;  t  e  J     ]  is  an  autoregressive  sequence  of  order  one, 
then 
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MSE^    -  MSZ^   =   ^~'^[k^{2)    -    (1  +  a^)    ^    a^  K^i4)K^    (2)} 

+   o(n"''"),  (2.2.31) 

\rtiere   h   (r)    is   the   r        cumulant   of   the  distribution  of    fZ    }    . 

Proof: 

Referring  to  (2.2.30),  we  must  evaluate 

(ii)   eTxT.  5^}  ,  and 
(iii)   Efx^.  6^6^}  . 

(i)   We  will  evaluate  the  first  term  in  detail  in  order  to  make 
the  methods  clear.   First,  note  that 

=    E[XJ  (N-^  £  Xl    -   y/] 

2  ?/  ^    2\2      -1     ^22 

=  ^f^   4(,f,  ^t)   -  ^^    YO  J^  <   < 

-  Yo  xj]  . 


(2.2.32) 


Now 

9  /  N     2     N  N  t-1    2   2   2 

Efx!  (  E  x;)  3  =   E  E[x;x}  +  2E   E  E[X  X  x  }  .   (2.2.33) 

^  \=1   *^      t=l    ^   ^      t=2  s=l    ^   s   t 

From  (2.2.16),  for  t  >  s, 

t-s-1 

t-S         I-  ^  J.    , 

X^  =  c    X  +    E   Q'  Z^  , 
t         s    ,  „      t-k 
k=0 

=  a~^   X  +  Y   ,  ,  (2.2.34) 

s    s ,  t 
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where 

t-s-1 


Y    =   E   d^  Z  .  (2.2.35) 

^'^    k=0      ^"*' 


We  now  write 


(k)      k 
^s,t  =  ^f^s.t^  '  (2.2.36) 


defining 

(k) 
V^  ^  =  Y    -  0  (2.2.37) 

s ,  s    s ,  s  ' 

for  all  k  e  J   ^.   Hence  we  obtain  from  (2.2.34),  for  t  S  N, 

=  «^«-«  EfxJ)  .  v<2>  EtxJ)  .  (2.2.38, 

Furthermore,  for   s  ^  t  S  N, 

EfxJx^xJ)  =  E[E[X^X^xWl) 

=  a^^-^EClc**-''  X  .  V   ]^  X^] 
s    s,  t    s-" 

t  ,N   *•         s    s,  t    s-" 
^^2(N-t).4(t-s)     6 

.  [6a2(N-t).2(t-s)  ^(2)    2(t-s)  (2)      4 
-  s,t  t,N    "■  s-" 

,  4^2(N-t).(t-s)  ^(3)  ^3 
s,  t   *■  s-" 

,.  2(N-t)   (4)     (2)   (2),   ,  2, 

Before  we  can  write  (2.2.32)  in  terms  of  the  cumulants  of  {Z  },  we  need 
to  note  that  the  r    cumulant  of  the  [X  }  process  is,  using  (2.2.17), 
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><^(r)  -  (1-/)'^  H^Cr)  ,  (2.2.40) 


and  hence  that 


EfX®3  -  a-a^)    ^    K^(6)  +  15[(l-a^)(l-a^)]"^  k  (4)  k  (2) 

+  10(1 -a^)"^  4(3)  +  15(1 -a^)"^  K^(2),         (2.2.41) 


E[X^}  =  (l-Q-^)  ^  K2(4)  +  3(1 -a^)  ^  H2(2),  (2.2.42) 


E{X^}  =  (1-cy^)"^  H^(3),  (2.2.43) 

s  ^ 


and 

EfX^}  =  (1-a^)"^  H,(2).  (2.2.44) 

We  will  also  need,  from  (2.2.35)  and  (2.2.36), 

/..^    t-s-1   ..  _         t-s-1   i-1   _,.  .,  _ 

V<^i  =    E   a'*^  [H ,(4)  .  3.2(2)]  .  6    E     E  a'^^^^^'(2) 
^•*    j=0         ^        ^  i=l    j=0  ^ 

=  d-C^-'  (l-c.^^*-^^K^(4) 

+  3(l-c.2)"2(l-a2^*"^^2  H2(2),  (2.2.45) 

V<^>  =  'T'  a^J  K  (3) 

=  (l-a^"^(l-a^^*"^^  K^O),  (2.2.46) 

and 

/ON       t-s-1 

vf   =    E   c^J  H^(2) 
®'^    j=0       ^ 

=  (l-a^"\l-a^^*"^^  K2(2)  .  (2.2.47) 

We  are  now  prepared  to  obtain  (2.2.39)  in  terms  of  the  cumulants  of 

{Z  };  the  result  of  substitution  of  (2.2.41)  -  (2.2.47)  into  (2.2.39)  is 
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^,,^2  J2.      2,    _   6-1   2(N-s)  +  2(t-s)    ,_. 

r.-.   4^^1   2.-2^^  2  (N-s)+2  (t-s) 
+  [(1-Q'  )  (l-Q-  )]   [Sa 

^  2(N-s)     2(N-t)     2(t-s)^    .^.    ... 

/■I   3  -2_  2(N-s)+2(t-s)    ^  2N-s-t.,   2 
+  (1-a  )   [eck-  +  4a      ]  H  (3) 

^-1   2,-3       2(N-s)      2(N-t) 
+  (1-a  )   [1+lOa      +  2a 

+  2a^^*"^'*]  >i\{2)    .  (2.2.48) 

We  now  substitute  (2.2.48)  into  (2.2.33)  to  get 

N  N-1 

EfxJ  (  Z  X^)^]  =  (l-a^)~^  K^iG)    [   E  a^ 
t=l  t=0 

+  2  E   E  a     ] 

t::rl  S=  1 

N-1   t 
+  [(l-a^)(l-a^)]"^H  (4)k  (2)fN+l6  E   E  a^^"^^* 

t=l  S=:l 

^-2   2t  2  ^"^   2t 

+   E  a  [2(N-t-l)  +  a  (2N+8t)]  +  14  E  a  } 

t=0  t=0 

3-2   2       ^"^   2t      ^"^   ^    2t+2s 
+  (1-a  )  ^  4(3)  {10  E  a^  +  12  E   E  a^^^^ 

t=0         s=2  t=l 

N-1  t-1 
+  8  E   E  a   } 
t=l  s=0 

2-3   3    ,2       ^"^   2t 
+  (1-a  )    K„(2){N  +2N  +   E  a   [4(N-t-l) 

^  t=0 

+  a^(4N+16t)]].        (2.2.49) 
The  second  term  on  the  right  side  of  (2.2.32)  is  similarly  evaluated  to 
obtain  in  part 


20 


NX.  Z 

t=l  t=0 

2-2   2         ^~J-      2(N-t) 
+  (l-a  )    H^(2)[l  +  2   E  a 

4  -1       ^-^   2t 
=  (l-a  )    K_(4)   E  a 

^      t=:0 

N-1 
+  (l-a^)~^  H^(2)  [N+2   E  a^*]  .  (2.2.50) 

^  t=0 

Putting  (2.2.49)  and  (2.2.50)  into  (2.2.32)  yields 

E{X^  6q}  =  N"^{[(l-cy^)(l-a^)^]"^(l+a^)  K^(4)    k^(2) 

+   2(l-cy^)"^(l+a^)  K^(2)}  +  o(n"^).  (2.2.51) 


(ii)   We  proceed  in  the  manner  described  above  to  obtain 
E{xJ  6^}  =  N"^[2[(l-o^)(l-a^)^]"^  a^K2(4)  K^(2) 

+  (l-cy^)"^(l+4cy^-a'*)  H^(2)]  +  o(n"^).  (2.2.52) 


(iii)  Similarly,  we  find 

EfX^  6^6^}  =  N"^f[(l-c^'*)(l-a^)^]"^  ail+a^)    k^(4)    K^i2) 

+   4(l-a^)"'*  a  K^<.2)]+o(}i~h.  (2.2.53) 

Finally,  from  (2.2.30)  and  (2.2.51)  -  (2.2.53), 

MSE^  -  MSE^  =  (1-a^)  K^^(2)  [E[X^  6^}  -  2aE[X^  6^6^} 

=  n"^[h2(2)  -  (l+Q-^)"^  a^  K2(4)  ^2^(2)}  +  o(n"^), 

((2.2.31)) 

which  completes  the  proof  of  the  theorem. 
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Note  that  if  [Z  }  is  a  normal  process,   K  (4)  is  zero,  so  that 

t  Ci 

MSE_  -  MSE   =  n"'''  k„(2)  +  o(N~  ) 

=   n'"*"  a^  +  o(n"-^).  (2.2.54) 

Thus  we  have  the  rather  remarkable  result  that,  after  [Z  }  is  a  normal 
process,  X   and  a   are  uncorrelated  when  terms  of  lower  order  than  N 
are  ignored.   That  is, 

MSE^  -  MSE^  =  E{xf]  Ef(^-Q')^}  +  oCn""^).  (2.2.55) 

2       IN  ■' 


Case  2:  p  =  1,  m  general 

Using  (2.2.9)  and  (2.2.16) 

i^^\m)  =  E[V  .J  Z^^^_.  .  .■"  V^^J  --y^'   ^,N^) 

=  aV,.  (2.2.56) 

N 

Then 

=  c\il-c^'^)a-c^)~'^    .  (2.2.57) 

Assuming  a  unknown,  one  forms  the  predictor 

i^^\m)  =  a""  x^.  (2.2.58) 

As  in  Case  1, 


MSE   -  MSE   =  E[[Xy^m)  -  X^^''(m)]''] 
z       J.       fj         N        ■' 


=  E[X^  [(5cy  +  a)""  -  a""]^],  •     (2.2.59) 


writing  ^c/  -   a  -   oi. 
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Upon  expanding  (2.2.59)  and  remembering  that  Sq-  has  sampling  error  of 

-1/2 
order  N    ,  we  have 

MSE^  -  MSE^  =  EfX^[(6c^)'"  +  m(y(6a)'""^+  ...  +  ma"'"^(6a)  ]^} 

2  2(ni— 1)    2    2       —1 
=  m  or      E[X^(.6q!)    }  +  o(N  ).  (2.2.60) 

Using  Theorem  2.2.1  and  assuming  Z   normal,  this  becomes 

MSE^  -  MSE^  =  N'Vc^^^^^'^^a^  +  o(n"^).  (2.2.61) 

If  the  assumption  is  dropped,  the  fourth  order  cumulant  must  be  included 
in  (2.2.61). 


Case  3:   p  general ,  m  =  1 

The  optimal  predictor  here  is 

^'^^'^  =  "  tfl  "*  'N.l-f  <2.2.62) 

with  prediction  error 


MSE 


i  =  ^ft\>i- V  <^^^  ^ 

P  2 

=  E{(X  ^  +  E  a.  X^^  ^  .)  1 
J*+l    .  j^   J   N+l-j   J 

2 

=  O^.  (2.2.63) 

The  predictor  becomes,  when  the  parameters  are  unknown, 

\   CD  =  -   E  a.    X  (2.2.64) 


j  =  l 


j   N+l-j 


The  increase  in  prediction  error  is  given  by 


MSE_  -  MSE 


.-(2),,,    -(1) 


-2   ..-^^  =  EfCX^-^l)  -  X^^^l)]^] 

=  £{(60-)'^  X  X^(6a)},  (2.2.65) 
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T 
using  matrix  notation  with  X  -    (X:,,,X,,  ...•■•  .^^r    )•  The  evaluation  of 

^  ~      N   N-1      N-p+i 

(2.2.65)  is  intractable;  Theorem  2.2.1  leads  us  to  conjecture  that,  for 
normal  Z  ,  X  and  (6q')  are  uncorrelated  (ignoring  terms  of  order  less 
than  N  ) .   In  this  case 

MSE^  -  MSE^  «Yo^C<^)'^  ^(^>}- 

T 
where  X  X  has  been  replaced  by  its  expected  value.   It  follows  from  the 

results  of  Mann  and  Wald  [21]  that,  asymptotically, 

V5r  (6a)  ^Np[cD  .g^CyqE)"^},  (2.2.66) 

where  N  denotes  a  p-variate  normal  distribution  and  cp   a  null  vector  of 
P  ^P 

order  p.   Thus 

MSE^  -  MSE^  =  Yol^^^^"^  4  P^  ^  o(n"^) 

=  n""*^  P    a1   +   o(n""^),  (2.2.67) 


T   2      -1-1 


using  the  fact  that   N(5g)  [a^(y   T.)      ]  (Sa)  has  a  chi-square  distribution 


with  p  degrees  of  freedom.   Note  that  expression  (2.2.67)  is  a  simple 
generalization  of  expression  (2.2.54)  for  the  case  p  =  1. 

Case  4:   p,  m  general 

In  this  most  general  case,  the  predictor  is  (2.2.10),  where 

the  values  x    (j)  are  determined  recursively.   When  fo'  ;  j  e  J    ]  are 

^  j        I.P-^ 

a(2) 

substituted  for  [q-.  ;  j  =  J    ],  one  obtains  x^   (m)  ,  given  by  (2.2.12). 

We  are  interested  in  calculating 

p 

MSE   -  MSE  =  Ef[  E  (a.  xf^^\m)  -  a.    xf^^\m))]^].         (2.2.68) 
■^       J-       .  ^   J   M  J   N 

All  attempts  to  obtain  (2.2.68)  in  closed  form  have  been  unsuccessful. 
That  the  term  is  of  order  N   follows  from  the  properties  of  (^a) . 
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Further  conclusions  might  be  possible  with  the  aid  of  Monte  Carlo 
experiments. 

In  summary,  the  autoregressive  process  allows  one  to  calculate 
estimators  having  many  satisfying  properties,  including  relative  ease 
of  calculation.   Furthermore,  the  effect  of  these  estimators  on  the 
error  of  prediction  becomes  negligible  when  N  is  large.   Exactly  how 
large  N  must  be  is  dependent  upon  p  and  m.   We  have  obtained  precise 
expressions  for  the  increase  in  prediction  error  for  several  special 
cases. 

2. 3  The  Moving  Average  Sequence 

If  fx^;  tej.     )]isa  stationary  time  series  satisfying  the 

*•   "t  (—00  CO   -*  -     w 

relationship 

X  =  Z  +  0  Z    +  ...  +  3  z  (2.3.1) 

t     t     I  t-1  q  t-q 

for  every  t,  then  [X  ]  is  said  to  be  a  q   -order  moving  average  process. 

The  problem  of  parameter  estimation  for  the  moving  average 

sequence  is  complicated  by  the  fact  that  the  sample  covariances,  C,  do 

not  provide  efficient  estimators  of  the  covariance  function  of  fX  ] ,  v  • 

<■  t-'   'j 

(Efficiency  is  defined  with  respect  to  the  variance  of  the  maximum 
likelihood  estimator  when  Z  is  assumed  normal.)  The  following  is 
a  brief  history  of  the  estimation  problem. 

Whittle  [33]  obtained  maximum  likelihood  equations  for  the 
moving  average  process  assuming  Z   to  be  normal.   Estimators  were  not 
obtained  from  these  equations,  however;  even  iterative  methods  have 
proven  unsatisfactory  for  solving  the  likelihood  equations.   The  search 
thus  began  for  estimators  of  calculable  form  and  with  high  efficiency. 
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We  note  that 
^J     ^  t+j   t-" 

q 


=  Ef(  Z   0^  Z^  .  ,)(  E   0^  Z.  .)} 

k=o  ^  *+J-^  k=o  ^  *-^ 


p  q-J 


if  j  e  J 


0,q 


if  j  e  J 


q+1, 


(2.3.2) 


defining  3„  =  1   and  noting  that  v  =  v  .•   One  can  obtain  estimates 
for  fP.;  j  e  J,   ]   by  substituting  C.  for  y.  in  (2.3.2)  for  j  e  J, 

J     Lq-*  J    ^j  i.q 

However,  the  lack  of  efficiency  of  C.  for  y.    has  a  predictable  effect 
on  these  estimators.   Whittle  [33]  has  shown  that  for  q  =  1  and  3  =0.5, 
the  efficiency  of  this  method  is  about  0.3. 

Another  representation  of  [X  ]  is  the  following: 


\  +      S  Q-.  X^  .  =  Z 
'    3  =  1      '      ^-'  ' 


(2.3.3) 


where 


Js^° 
Ej  =r 


\'i 


q 


(2.3.4) 


and  where  fl] .  ;  j  e  J    }  are  the  (distinct)  roots  of 


z^  +   3  z^  ^  +  ...  +  3  =  0. 
1  q 


(2.3.5) 
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If  the  roots  of  (2.3.5)  are  not  all  different,  the  representation  is 
not  affected;  only  the  definition  of  the  parameter  set  [a.;    j  e  J    } 
differs  slightly.   Also,  in  order  that  (2.3.3)  be  a  valid  representa- 
tion (in  the  mean  square  sense) ,  we  must  require  that  all  roots  of 
(2.3.5)  lie  inside  the  unit  circle.   This  restriction  is  not  serious; 
in  fact,  it  insures  that  the  representation  (2.3.1)  is  unique. 

Thus  fX  }  may  be  considered  an  autoregressive  process  of  infin- 
ite order.   Durbin  [10]  proposes  that,  since  it  is  easily  shown  that 
»  decreases  exponentially  in  r,  we  use  the  representation 

k 

\+      2  a  X    ;:«  Z  (2.3.6) 

t    .^^     J      t-j     t 

where  k  is  chosen  large,  but  still  k  «  N.   The  parameters  [0-  ;  j  e  J    ] 

may  then  be  estimated  using  methods  discussed  in  2.2;  finally,  (2.3.4) 

and  (2.3.5)  are  used  to  transform  [5  ;  j  e  J,  ,  ]  to  estimators  of 

J       l.k-' 

{p.;  j  e  J    ].   Durbin  shows  the  estimators  can  be  made  efficient  by 

J     i.q 

choosing  k(and  hence  N)  large.   All  properties  derived  assume  N  infinite 
and  k  large  enough  that  relationships  (2.3.3)  and  (2.3.6)  are  nearly 
equivalent.   The  computation  of  the  estimators  requires  considerable 
effort. 

Walker  [31]  finds  linear  functions  of  the  sample  correlations 
{r  =  *^T  ^0  '  -^  ^  "^1  N^  which  provide  asymptotically  efficient  esti- 
mators of  the  correlation  function  [p.  =   y.    v"  >  j  e  J    ].   He  then 

J     J   ^         1 1 Q 

substitutes  these  estimators  into  (2.3.2)  (expressing  (2.3.2)  in  terms 
of  correlations  rather  than  covariances) ,  obtaining  asymptotically 
efficient  estimators  of  [p . ;  j  e  J    }.   The  efficiency  depends  pri- 
marily on  the  number  of  correlations  used  in  the  calculation  of  the 
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estimators  and  the  number  of  observations,  N.  The  weights  assigned  to 
the  sample  correlations  used  in  these  linear  estimators  are  determined 
iteratively,  so  calculations  may  be  tedious. 

The  most  recent  contribution  is  due  to  Hannan  [16].   He  uses 
the  spectral  representation  of  {X  }  to  obtain  parameter  estimators. 
Though  his  work  is  too  complicated  to  summarize  here,  it  should  be 
noted  that  the  estimators  are  shown  to  be  asymptotically  normal  with 
variance-covariance  matrix  identical  to  that  of  the  maximum  likelihood 
estimators.   The  procedure  does  involve  an  iterative  procedure  which, 
by  his  own  examples,  may  be  slow  and  thus  expensive.   His  Monte  Carlo 
experiments  also  showed  inexplicable  bias  in  his  estimates. 

By  now  one  realizes  that  the  estimation  problem  for  the  moving 
average  time  series  is  anything  but  simple;  but  the  author's  objective 
is  to  obtain  an  estimator  suitable  for  use  in  a  predictor.   The  predic- 
tor which  minimizes  the  mean  square  error  of  prediction  is 

x^^m)  =  e[x   /(X.  =  x.;  j  e  J^  J.  (2.3.7) 

N         *•  N+m   J     J        1,N-' 

If  the  entire  history  of  [X  }  were  known,  that  is,  if  the  observation 

were  fx.;  j  e  J   „] ,  we  would  have 

.(1)         "^ 

xA  ^(m)  =  E[  E  g.  Z^^    ./(X.  =  X.;  j  e  J    )].  (2.3.8) 

i=0  -^   N+m-j    J     J  '  -"     -00, N  J 

Note  that  [X.  =  x.;  j  e  J_^   ]  uniquely  determines  f Z .  =  z . ;  j  e  J_^   }, 

and  conversely,  by  using  (2.3.1)  and  (2.3.3).   Thus 

EfZ^^  ,/(X.=x.;jeJ     ] 
••  N+k   J     J  '  -"     -»,N-' 


=  EfZ  ,/(Z.  =  z.;  j  e  J     } 
'•    N+k    J     J  '  "^     -=°,N-' 


J     J 
=  0,  (2.3.9) 
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for  k  e  J    ,  since  the  fZ  ]  process  is  an  uncorrelated  sequence, 
1 ,  00  ^  t  -^ 

Thus  (2.3.8)  becomes 

.(1)       "^ 

x„   (ra)  =   E  p.  z^^    .  .  (2,3.10) 

N  J   N+m-j 

J=ni 

Remembering  that  a     decreases  exponentially  in  r,  we  can  approximate 
r 

^N-k  ^y 

N-k-1 

^xr  V  «  ^XT  u  +     ^    Q--  '^M  ,,   •  (2.3,11) 

N-k    N-k     .  ,    J   N-k-j 
J  =  l 

for  use  in  (2.3.10).   Henceforth  we  assume  that  the  entire  history  of 

the  {X  ]  series  is  known  up  to  time  N  so  that  the  discussion  is  not 

needlessly  complicated. 

Case  1:   q  =  ra  =  1 

From  (2.3. 10) 


(2.3.12) 


-(1),,,         Q 

writing  0=0.   The  error  is 

=  ^f(^N.i  -  p  v':f 

=  a^.  (2.3.13) 

If  P  is  unknown,  and  one  uses  the  estimator  0,  we  have 

(2)  /v 

^  '(1)  ^  0  z^,  (2.3.14) 

where  from  (2.3.3) 

CO 

Z   =  X  +   Z   (-0)-^  X^  .,  (2.3.15) 

^   j  =  l         ^"^ 
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and  so 


From  previous  arguments  we  know  that 

MSE^  -  MSE,  =  E[Oz^^  -  0  Z^)^}.  (2.3.17) 

2       IN      N 

-1/2 
If  we   assume   that    3  has   sampling   error   of   order  N  ,    which 

is   the   case   for   each   of   the   estimation  procedures    considered   above, 

we  have 


MSE, 


-  MSE^    =   E[[3Z^   -    (63+p)(6z  +Z    )]    } 
IN  N     N         ■" 


-1, 


=  EUza^r]   +    3      E[(6Z   )    ]   +   o(N   "),  (2.3.18) 

^      N  ■'  N 


where      63  =    0   -  p      and      6z^,  =   Z^^  -   Z^  . 

N  N  N 


Not 


e   that    it    follows    from    (2.3.9)    that    if  we  use    fx.;    j    e  J^    ^^   ^ } 

J       1,N-1 


in  calculating  3,  then  3  and  Z   are  uncorrelated.   Then 
E{Z^(63)2}  =  E[Z^}  Ef(63)^} 

=  al   E{(3-3)^}  .  (2.3.19) 

The  second  term  on  the  right-hand  side  of  (2.3.18)  follows  from 
E{(6Z  )2]  =  E[[  E  (-1)^  (3^'-3^)X   ]2] 

CO 

=  E{[  E  (-l)'^  j  3'^"^(63)X„  .]^}  +  o(n"^).       (2.3.20) 
j  =  l  ^-J 

The  value  of  (2.3.20)  depends  on  the  specific  form  of  3.   However,  the 

expression  will  be  complicated  for  all  estimates  considered.   One  can 
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2 
obtain  an  approximation  for  (2.3.20)  by  replacing  (5P)   by  its  expected 


value  to  get 


J  =  l 

00 

-  2y.   E  j(j  +  l)3^^^"^^} 

=  Ef(63)2}  p^  CT2(l-3^)"^  (2.3.21) 

Substitution  of  (2.3.19)  and  (2.3.21)  into  (2.3.18)  yields 
2   2_    „2_  n2.-l 


USE 


^   -  MSE^  ^  Ef(63)"  a^Ll  +  0  (1-P  )   ]} 


=  E{(P-3)^}  a^d-P^)  ^.  (2.3.22) 

Whittle  [33]  has  shown  that  the  maximum  likelihood  estimator  for  0  has 

—1     2 

asymptotic  v"ariance  N   (1-0  ),  so  that,  at  best,  the  increase  in  the 

mean  square  error  of  prediction  is  given  by 

MSE^  -  MSE   =  n"''"  a^  •  (2.3.23) 

A  strong  argument  can  be  made  for  using  the  inefficient  estimator 
given  by  substituting  sample  correlations  into  (2.3.2)  and  solving  for  3; 
the  estimation  equation  is  thus 

r^  =  C^  C^^  =  3(1+3^)"^  .  (2.3.24) 

Only  one  of  the  solutions  to  this  quadratic  will  be  less  than  unity. 
The  estimator  is  easy  to  calculate,  and  even  if  the  efficiency  is  only 
0.1,  the  additional  error  given  by  (2.3.22)  is  negligible  for  large  N. 

Case  2:   q,  m  general  (m  <  q) 

Using  the  m-step  predictor  given  in  (2.3.10),  the  mean  square 
error  is 
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q-m  2 

1  "■      N+m         ._        j+m     N-j       ■' 

=  otn  +    i:   3  )  .  (2.3.25) 

'     J=l  ^ 


TOien   the   estimator 

q 


3ci^\m)    =      Z     e.    5  (2.3.26) 

N  J      N+m-j 


is  used,  the  increase  in  prediction  error  is  given  by 


MSE^  -  MSE^  =  E[[X^  ^  -  X^  ^]  } 


**  2 

=  Ef[  E  [z^   .(63.)  +  3.(Sz    .)}]  } 

+   o(n"''").  (2.3.27) 

By  using  only  fx.;  j  e  J^  ,   ]  in  the  calculation  of  [3 . ;  j  e  J,   ], 
J        l.N-q-'  '•J        l,q-' 

the  cross  product  terms  in  (2.3.27)  vanish,  leaving 

q  q 

MSE„    -  MSE.,    =   e{[    E     Z^^         .(63.)]^}  +   E[[    Z        3.62^^         .]^} 
2  1  *•    .  N+m-j        J         •*  '-     .  J      N+m~j       -• 

J=m  ^        ^  j=m 

+   o(n"'^).  (2.3.28) 

The  first  term  on  the  right-hand  side  of  (2.3.28)  is  easily 

evaluated  as 

q  q 

E{  Z  Z^    .(63.)^]  =    ot      H     E[(3.-3.)^}.  (2.3.29) 

J=m  j=m 

The  second  term,  however,  is  complicated  by  the  fact  that  the  fX  } 

process  is  q-dependent;  that  is  to  say,  f  v .  !  J  e  J     ]  ^i"©  all  non-zero. 

J'  (-Yj   -J     _q^qJ 

If  we  apply  the  methods  used  in  Case  1, 

E{[  Z  3.  6Z    .]^}-E^   Z  3.  (  Z  (5.-cy.)X^^   .  jfj, 
j=m  ->   N+'^-J       ^^^m     J  H=l   J   J   N+m-j-t;jJ 

(2.3.30) 
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where  {q'.  ;  j  e  J   }  are  obtained  from  (2.3.4),  using  f "H .  ;  j  e  J    ] 
as  the  roots  of 

^   +  0-  z'^"''"  +  ...  +  3   =0.  (2.3.31) 

All  that  could  be  gleaned  from  (2.3.30)  is  the  fact  that  an 
increase  in  q  will  result  in  an  increase  in  the  prediction  error 
increase  given  by  (2.3.27).   But  since  no  more  concise  expression  could 
be  obtained  for  (2.3.30),  no  approximation  was  obtained  for  (2.3.27). 

The  argument  for  using  the  estimators  which  are  easily  calculated, 
but  perhaps  inefficient,  still  applies,  of  course.   But  determination 
of  N  in  order  that  the  increase  be  bounded  by  a  given  quantity  must  be 
accomplished   by  using  numerical  methods. 

2.4  The  Hybrid  Model 

If  f X^ ;  t  e  J    1  is  a  stationary  time  series  satisfving  the 
^  t       ~^ , ^  " 

relationship 

t     1  t-l  p  t-p     t     1  t-l  "^q  t-q    (2.4.1) 

for  every  t,  then  {X  ]  is  said  to  be  a  hybrid  model  of  order  (p,q). 

Estimation  of  the  parameter  set  [a^ o'  ;  B,  , .  .  .  ,  3  ]  for  the 

••  1      p    1       q-" 

hybrid  model  is  the  most  difficult  of  those  considered.   The  fact  that 
the  sample  covariance  function  is  an  inefficient  estimator  of  the  true 
covariance  function  contributes  to  this  difficulty.   The  maximum  like- 
lihood estimators  are  again  intractable,  but  their  asymptotic  variance- 
covariance  matrix  can  be  used  for  calculations  of  ef f iciencj''. 

Durbin  [11,12]  again  makes  use  of  an  infinite  order  autoregres- 
sive  representation  of  [X  ].   The  problems  with  the  procedure  are  similar 
to  those  mentioned  in  2.3:   the  choice  of  truncation  point  for  the 
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autoregressive  representation,  the  choice  of  N,  and  the  cumbersome 
iterative  procedure  used  in  the  calculations.   However,  the  efficiency 
of  the  estimators  can  be  made  as  close  to  unity  as  desired  by  taking 
the  truncation  point  and  N  large. 

Walker  [32]  approaches  the  problem  via  the  covariance  function; 
by  finding  an  efficient  estimator  of  this  function  one  can  efficiently 
estimate  the  hybrid  model  parameters.   The  following  relationships  are 

used: 

P  P 

E{(  E  Q-.  X^  .)(  H     a.   X      .)} 

q         q 

=  E{(  E  3.  Z   .)(  E  3.  Z   .)}  (2.4.2) 

implies  that 

P    P  2  ^ 

E   E  a. a.  y,^      .     ,.     ..  =CT     E    3.  P.  r    I,  (2.4.3) 

j^O  i=0   ^  J  ^(t-s)  +  (i-jO   Zj^|^_^jPj   J-|t-sl' 

where  cx„  =  3f^  =  1.  ^nd   |t-s|  S  q.   This  system  of  equations  gives 
\_0i.\    j  e  J,   }  and  fP.;  j  e  J,   ]  in  terms  of  the  covariance  function 

J     i,p      J     i.q-" 

{y.}-   In  order  that  the  solutions  to  (2.4.3)  be  unique,  one  must  require 
that  the  roots  of 

^   +  3^  z*'"-'-  +...+3=0  (2.4.4) 

have  modulus  less  than  unity. 

Also,  after  multiplication  of  (2.4.1)  by  X    for  r  e  J 

t-r  q+i,q+p 

taking   expected  values   yields 


2     a.    Yr_i    =0.  (2.4.5) 

j-0    -J     ^  J 
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using  the  fact  that  X   is  uncorrelated  with  the  sequence 

fZ.  . ;  j  e  J,   ].   Walker  finds  efficient  estimators  of  fv  ],  and  then 
t+j        l,*"-"  '  j 

uses  (2.4.3)  and  (2.4.5)  to  calculate  estimates  of  the  hybrid  model 

parameters.   An  iterative  procedure  is  required,  but  the  resulting 

estimators  are  asymptotically  normal  and  fully  efficient. 

Hannan  [16]  achieves  the  same  goal  by  a  different  method: 

that  of  estimating  the  spectral  density  and  using  this  to  get  estimates 

of  the  parameters.   The  effort  required  is  substantial,  but  the  esti- 

nators  are  asymptotically  normal  and  efficient. 

The  predictor  of  X,   given  the  observation  fx  ;  j  e  J    1  is 
N+m  "-J       -oo.N-^ 

xj^\m)  =  EfX   /(X.  -  X.;  j  e  J    )} 
N  *■  N+m    J     J        -".N  ■• 


*^      .(1)  ^ 

-     Z     a.    x;  ''(m-j)  +   E  3.  z^^ 


=  < 


if  m  e  J 


-  E  a  X   (ra-j) 
J=l  -^ 


if  m  e  J 


i,q 

q+l,»' 


(2.4.6) 


irtiere 


-.(1),.,    . 


Efx.,  ./(X.  =  X.;  j  e  J    ^ 


if  j  e  J 


1,00 


'^■ 


+3 


if  j  e  J 


-00,0" 


(2.4.7) 


Note  that  the  values  of  fz^,  .  ;  j  e  J,   1  are  calculated  from 

•■  N-j   "^     l,q-' 

{x.;  j  6  J  jg  „}.  using  the  infinite  order  autoregressive  representation. 
When  only  [x.;  j  e  J    ]  is  observed,  we  may  approximate  [z      .;  j  e J    } 
by  truncating  the  autoregressive  representation  after  N  terms. 
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When  the  parameters  are  unknown,  the  predictor  used  is 


3^-   (m)  =  < 


E  a.   x;/\m-j)  +  E-  3.  £    . 


if  m  e  J 


i,q 


E  a.  X   (m-j) 
j  =  l  ^  ^ 


if   m  e  J   ,   ,   (2.4.8) 
q+l,<n 


where  x    (j)  is  calculated  just  as  x    ( j )  with  the  parameters  replaced 

N  N 

by  their  estimators,  and  z.  is  calculated,  using  the  autoregressive  rep- 
resentations with  estimated  parameters. 


Case  1:   p  =  q  =  m  =  1 

The  time  series  fx  :  t  e  J     ]  satisf 


t-1 


writing  a  =   -a^    and 


t-1' 
in  (2.4. 1).   From  (2.4.6) , 


(2.4.9) 


5^^\l)  =  cyx.,  +  3z„. 


N     N 


(2.4. 10) 


The  autoregressive  representation  of  Z   is 

N 


Z„  =  X  +   E  (-^y    (X^^  .  -  aX^,  .   ) 
N    N    .  ,  N-j      N-j-1 

J  =  l 


=  Xj^  +  (1  -  aP  ^)   E  (-P)-"  X 
j  =  l 


N-j 


(2.4.11) 


Noticing  that  the  autoregressive  parameters  (the  coefficients  of  the 
X  ' s)  on  the  right-hand  side  of  (2.4.11)  are  decreasing  exponentially, 
we  will  use 


_   N-1 

z„  «  X  +  (1  -  aP   )   E   (-3)-^  X   . 
N    N  .  N-J 


(2.4.12) 
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in  (2.4.10)  to  calculate  x^   (m) .   The  mean  square  error  of  prediction 
for  (2.4.10)  is 


MSE 


l-^f^Vl-^N^'"^^  ^ 


.2. 


=  a^  •  (2.4.13) 

When  a   and  0  are  estimated  by  a   and  P,  respectively,  we  use 

x^^\m)  =  Sx^+  PzY  (2.4.14) 

The  increase  in  prediction  error  is 

,2. 


MSE 


2  -  MSE^  =  Ef[cyX^+3Z^-(5x^+aZ^)]  }.  (2.4.15) 


The  term  Z  greatly  complicates  this  expression;  but  if  we  assume 


Z  ^  Z  , 

■N    N' 


MSE^  -  MSE^  «i  E{[(5-<:^)X^  +  (g-p)Z^]^} 


+  2a^E[(5'-a)(3-0)3,  (2.4.16) 


where 


(i)  X   and  oi   are  assumed  approximately  uncorrelated; 
(ii)  (a,0)  are  calculated  using  only  {x.;  j  e  J   vr_-.  }  ^o  that 
3  and  Z   are  uncorrelated;  and,  using  the  fact  that  fx  }  can  be  repre- 
sented  as  an  infinite  order  moving  average  process, 
(iii)  E{X^Z^}  =  Ef[Z^  .  J^  ^jVj^^N^ 

=  dt.  (2.4.17) 

z 
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To  obtain  f 7]^  ;  j  e  j^^J  m  terms  of  (c^,3),  note  that 


X   -  7   _L    V   -J 

3- 


J-1 


=  Z  +  (c^3)  E  a^  ■"  z 
*        J=l      *-^ 


(2.4.18) 

One  can  derive  an  estimator  for  a   from  the  equation  (2.4.5): 

"^r  ~  ^r-1  =  °  (2.4.19) 

for  every  r  e  J^^^.   With  this  in  mind,  we  define  E^  as  an  asymptotic 
(kxk)  covariance  matrix  with  (i,j)  element 

where   C  =  C    C 

i    i+1   i  (2.4.21) 


for 


i  e  J^  j^.   By  letting 

fiC.  =  C.    C"-^  -  V    v"-^ 
1    i+1  i    Yi+i  Yi 


=  C,  -  a 


(2.4.22) 


and 


we  have 


5C.  =  C-  Y  , 

1    1    ''i  (2.4.23) 


=  (6c.^^-a^C)(6c  +Y)-1 


-  Y-i 


■(6C     -  Q^c.)  +  o  (n"^^) 
J  +  l      J      p 


(2.4.24) 


But  Bartlett  [1]  has  shown  that  C.  is  consistent  for  y.,    so  that 

(2.4.24)  implies  that  C*  is  consistent  for  a 
3 


Furthermore,  from  (2.4.18)  we  find 

if   s  =  0 


Y«  =  < 


Q^a+?^+2a3)a-a^)    ^ 


2   m-1 


if  s  ^  0, 


where 


a  =  (cw-P)(l+a3)(l-a  )~^   • 


Defining 


y      =  E{(6C  )(6C   )}, 
'T,r+s    '■   r    r+s  -' 

Bartlett  [1]  has  shown  that 

00 


)  +  o(l) 


J= 


-1 


Remembering  that  p.  =  Y-Yn  '  *^  "^^  (2.4.25)  to  get 
E  p  .p  .   =  / 


2     2-1 
1  +  2a  (1-a  ) 
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(2.4.25) 


(2.4.26) 


(2.4.27) 


(2.4.28) 


if   s  =  0 


J=- 


We  now  state 


a^   ^[2a^a^(l-Q'^)  ^  +  (k-1)  a^  +  2aQ']    if  s  ;:^  0. 


(2.4.29) 


Theorem  2.4.1: 


The  estimator 


»o  =  .f ^  "jo  <=j  • 


(2.4.30) 


where   d^  =  (^o'^20  '  '  "  "  '^kO^ 


T   -1    T   -1    -1 


and 


^0  1=^ 


(2.4.31) 


(2.4.32) 
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with   Tl   :r  (1,1 1)    ,  is  consistent  and  has  minimum  mean  square 

-L  IXk' 

error  among  the  estimators 


5  =    i:    d.  c. 


(2.4.33) 


with   d  T)  =  1.   This  error  is  given  by 


mse(Oq)  -  [Tf  e"^  V  ^, 


(2.4.34) 


where 


a      =  < 

r  ,r+s 


(ao'   )    d+Q'  -  4aQM-2a  )     if   s  =  0 


r  -2       2   2  2 

(aa   )    (2aQ'-Q'  -a  a  ) 


a         (l-Q-  ) 


if   s  =  1 

if  s  e  J 


2,00     (2.4.35) 


Proof : 

That  a     is  consistent  for  a   follows  immediately  from  the  fact 

*  T  _ 

that  C.  is  consistent  for  a   (2. 4".  24)  and  cl„  T^  =  1. 
J  -0  -i- 

Now  if  a   is  of  form  (2.3.33), 

MSEfa}  =  d^  S  d^.  (2.4.36) 

T 
In  order  to  minimize  this  with  the  restriction  d  T|_=:  1,  we  form  the 

Lagrange  equation 


L  =  d"^  Z  d  -  2X(cl'^  Tl  -  1) 


(2.4.37) 


But     -:rTl     =  0  implies 


T         T 

^Q  E  -  X  TT  -  0. 


(2.4.38) 


40 


T 
Using   d      T]  =    1,    it    follows    that 

\  =    [Tp^   e"^  T\)~^  (2.4.39) 

and  hence 

dj  =   f   E'^[Tf  e"^  T]]"\  (2.4.40) 

as  was   to  be   sho\vn. 

Substituting    (2.4.40)    into    (2.4.36),    we   obtain 

MSE[5q}   =    [if   E"^  Tl)''^.  (2.4.41) 

Finally,  noting  from  (2.4.24)  that 

a      =  Ef(6C*)(5C*   )} 
r,r+s     "•    r    r+s  ■' 

=  WrVr+s^'^^^+s,r+s+l-  ^'^^.r+s+l  "^  Vl,r+s^ 

+  ^\^r+s^'  (2.4.42) 

substitution  of  (2.4.28)  and  (2.4.29)  into  (2.4.42)  yields  (2.4.35), 
which  completes  the  proof. 

Table  2.4.1  gives  efficiencies  for  a  for  several  values  of 
0/   and  P;  three  values  of  k  (the  number  of  sample  covariances  used  in 
the  calculation  of  a  )    are  given.   All  calculations  were  carried  out 
on  the  University  of  Florida's  IBM  360  computer.   Also,  efficiencies  are 
calculated  with  respect  to  the  variance  of  the  maximum  likelihood  esti- 
mator of  Q'  given  by  Whittle  [appendix  of  35]  as 

W  =  N"''"(l-a^)(l+a0)^(cyfp)"^.  (2.4.43) 

The  table  shows  that  the  efficiency  is  especially  sensitive  to 

changes  in  B;  as  0  tends  to  unity,  it  becomes  necessary  to  use  more 

sample  covariances  to  achieve  high  efficiency. 
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k  =   4 


Table  2.1 


Tables  of  Efficiencies   for  a„  =      T,     d.„   C 

0         .    ,      jO      J 


j  =  l 


=    2 

a 

.1 

.3 

.5 

.7 

.9 

.1 

0.999 

0.999 

0.999 

1.000 

1.000 

.3 

0.952 

0.961 

0.972 

0.983 

0.994 

.5 

0.791 

0.840 

0.888 

0.934 

0.979 

.7 

0.591 

0.687 

0.780 

0.871 

0.958 

.9 

0.427 

0.551 

0.680 

0.809 

0.937 

4 

a 

.1 

.3 

.5 

.7 

.9 

.1 

1.000 

1.000 

1.000 

1.000 

1.000 

.3 

0.999 

0.999 

0.999 

1.000 

1.000 

.5 

0.967 

0.975 

0.983 

0.990 

0.997 

.7 

0.830 

0.877 

0.918 

0.954 

0.986 

.9 

0.632 

0.731 

0.819 

0.897 

0.968 

k  =   6  Q. 

.1  .3  .5  .7  .9 

.1  1.000  1.000  1.000  1.000  1.000 

.3  1.000  1.000  1.000  1.000  1.000 

.5  0.996  0.997  0.998  0.999  1.000 

.7  0.931  0.951  0.968  0.982  0.995 

.9  0.746  0.820  0.882  0.935  0.980 
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In  practice,  one  would  need  to  estimate  3  (using  2.4.2)  in  order 

to  calculate  a   .      Thus,  though  the  effort  of  calculation  is  somewhat 
o 

reduced,  an  iterative  method  must  be  used.   We  can  obtain  an  upper 
bound  to  the  mean  square  error  of  prediction  for 
>(2) 


Xj^  •  (m)  .  Cq  x^  4-  P  z^ 


(2.4.44) 


by  considering 
.(3) 


^  -  (n^)  =  ^0  ^N- 


(2.4.45) 


The  corresponding  increase  in  prediction  error  is 


MSE 


3  -  MSE^  =  E[[(ao-a)X^  +  PZ^]  } 


a^CS^  +  K  ^a+c3)^], 


(2.4.46) 


where  we  assume  K  observations  [x.;  j  ^  J    }  have  been  used  in  the 

calculation  of  a  ,  with  K  «  N,  in  order  that  the  assumption  that  X 

and  Q-  are  uncorrelated  is  more  plausible. 
0 

'^(3) 

In  fact,  the  use  of  x    (m)  as  a  predictor  of  X„   may  not  be 

N  N+m 

umreasonable  as  long  as  0  is  small ,  thus  saving  the  rather  tedious 

calculation  of  z^^. 

N 


Case  2:   p  =  q  =  1;  m  general 

The  m-step  predictor  is  given  by 


x„   (m)  =   a     X  +  0-   Pz 

N  N  N 


(2.4.47) 


with  mean  square  error 


MSE. 


if  m  =  1 


2-1. 


a2[l+  (c^8)^(l-a^'")(l-a")  ^]    if  m  e  J 


2,»" 


(2.4.48) 
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If  we  use 

Xj^^\m)  =  ^"'x^  +  S"'"^  0  z^,  '   (2.4.49) 

the  increase  in  prediction  error  will  be 

^m-1  ;^  m-l 


MSE. 


-  MSE,  ^E[l(a"-a)X^+(a     "  0-a"  "  3)2  1"] 
1     ^  N  N   ■' 

=  E{[inc^™"^6cy)X^+  (^"""^(60) 

+  ma"'"^  0(6a))Zj^]^}  +  o(n"^) 

2  2(m-l),    „2  2,^r/x  x2-,     2  (m-l)^,  .kqn  2. 
«  m  Q-      Wo+^  a^)E[(bQ!)    }  +  0-      E{(60)  } 

+  2ma^^'""^\l+3)a^  E[(6a)(60)},  (2.4.50) 


where  ue  have  replaced  Z„  by   Z„,    and   assumed    (5,0)    to  be  uncorrelated 

N     N 

with  X^. 

We  can  again  obtain  an  upper  bound  for  this  rather  complicated 
expression  by  considering 

4^\in)  .  ^  x^.  (2.4.51) 

Then  JISE      -  MSE      is  bounded   above  by 


MSE3   -  MSE^  .    E{[5^^   -    (a\  +    J"-^   0Z^)]^} 


^  Cj2  ^2(m-l)^p2   ^  K"^m(l+a0)^].  (2.4.52) 

In  addition  to  the  comments  applied  to  this  quantity  in  Case  1,  notice 
that  the  increase  becomes  less  significant  as  m  is  increased. 
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Case  3:   p  general;  q  i=  m  =  1 

The  representation  of  f X, ;  t  e  J     1  is  now,  from  2.4.1, 

^      t  — CO    ro-' 

X.    +    a^X.    ^    +    ...    +   a  X.         =    Z     +    pZ  (2.4.53) 

t  1   t-1  p   t-p  t  t-1 

The  predictor    is 

(1)  P 

x;   "^(l)    =    -      E     a.    x^,   ^     .    +    pz^.  (2.4.54) 

N  •_!      J      N+l-j  N 

and  the  error  of  prediction  is 

P 


MSE^  =  ^[[^N.l  -  ^.^,  ^J  Vl-J  '    P^N>^  3 


(2.4.55) 


Wlien  fQ'.,6;  j  e  J,   ]  are  unknown,  estimators  can  be  obtained 
"•J         1 ,  P  -^ 

from  (2.4.2)  and  (2.4.5);  we  have 

P 

E  Q-.  C   .  =  0  (2.4.56) 

for   r  e  J„      (o"  =  1)  and  (2.4.3)  for  estimating  the  parameters. 
2,p+l   0 

One  can  also  extend  the  method  presented  in  Theorem  2.4.1  to  this  case, 
but  the  analysis  is  increasingly  complicated  as  p  increases.   The  deter- 
mination of  estimates  from  (2.4.56)  and  (2.4.3)  is  probably  easiest, 
and  the  order  of  the  error  increase  is  N   ,  even  thougli  the  estimators 
are  inefficient. 

The  cases  with  p,  q,  and  m  general  are  analytically  intractable 
insofar  as  evaluation  of  increase  in  prediction  error  due  to  parameter 
estimation  is  concerned.   But  the  study  of  the  simple  cases  has  shown 
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that  primary  consideration  should  be  given  to  the  construction  of 
estimators  which  are  consistent  and  have  mean  square  error  of  order 
N   .   As  long  as  N  is  large,  high  efficiency  is  not  a  necessary 
condition  for  small  mean  square  error  of  prediction. 


CHAPTER  III 
THE  LOGISTIC  GROWTH  PROCESS 

3. 1  Introduction 

Let  fx(t) :  t  e  I„   ]  be  a  time  series  such  that 
0 ,  °° 

E{X(t)/X(0)  =  n}  =  n(l+cy)(l+cyp*)"^  ,  (3.1.1) 

where  a  >  0,  n  e  J    ,  and  n  e  (0,1).   Then  fX(t)]  is  said  to  be  a 
1,<=  ■' 

growth  process  following  the  logistic  law,  or  simply  a  logistic  growth 
process.   This  process  receives  wide  application  in  animal  population 
models  and  various  economic  models;  the  representation  varies  slightly, 
depending  upon  the  particular  application,  but  with  a  little  adjustment 
each  can  be  written  as  (3.1.1). 

The  problems  to  be  considered  are: 
(i)  the  estimation  of  ex   and  p  in  (3.  1.  1)  ,  and 

(ii)  the  prediction  of  future  values  of  X(t)  when  a   and  p 
are  estimated. 

Several  observational  possibilities  will  be  considered.   The 
process  may  be  observed  periodically,  at  equidistant  time  points;  it 
may  be  observed  continuously  during  a  given  time  interval;  or  only 
the  times  at  which  the  process  increases  (assuming  all  increases  are 
discrete)  may  be  observed. 


46 


47 


3.2  Probability  Distribution  of  fxCt)] 

The  probability  structiire  of  a  growth  process  may  be  specified 
in  any  cf  an  infinite  number  of  ways.   But  in  reality  these  processes 
are  often  used  wi-h  no  reference  to  such  a  structure;  the  parameter 
estimation  proble-  is  considered  without  reference  to  error. 

Ve  shall  use  the  following  probabilistic  model  for  [X(t)}: 

let  fX(t) :  t  £  I^   ]  be  a  Markov  process  with 
0 ,  ™ 

PfX(t+5t)  =  j/X(t)  =  i] 

iX(t)6t  +  o(5t)     if  j  =  i  +  1 
=<1  -  iX(t)6t  +  o(6t)  if  j  =  i 

o(6t)  otherwise,  (3.2.1) 

where  £t  is  an  infinitesimal  time  interval.   This  model  obviously 
requires  the  assumption  that  [X(t)}  grows  in  equivalent  discrete  units 
(econo-nic  units,  population  units,  etc.),  and  that  the  probability  of 
a  one-vmit  increase  at  any  given  time  is  proportional  to  the  size  of 
the  process  at  that  time.   The  reader  will  recognize  (3.2.1)  as  the 
common  definition  given  a  birth-death  process  with  the  death  rate  zero 
for  all  t.   Kendall  [19]  has  shown  that  the  above  representation  is 
the  birth-death  r.odel  which  minimizes  the  variance  of  {X(t)}. 

Defining 

p..{s.t)  =  P{x(t)  -  j/X(s)  =  i],  (3.2.2) 

an  application   of    the  Kolmogorov   equations    yields 
■  (J-l)?.(t)6t   p.     ._^(s,t)  + 

+  [1  -  jX(t)6t]p.  .(s,t)  +o(6t)        if     j    e  J 

if     j   =   i. 


p.  .(s,t+5t)   =  < 


[l-iX(t)6t]p.  .(s,t)  +o(6t) 


(3.2.3) 
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Thus 


■^   [p. .(s,t)]  = 


(j-l)X(t)p^   .l^s.t)  -  jX(t)p.  .(s,t) 
if  j  e  J.  , 

1+1,00 


-JX(t)p.  .(s,t)       if  j  =  i. 


(3.2.4) 


Multiplication  of  (3.2.4)  by  j  and  summation  over  all  j  gives 


_i:_  j[^fp..(s,t)}]  = 


=   X(t)[   S   j(j-l)p.  ._,(s,t)  -   E  j  p..(s,t)] 
j=i+l        ^'-^  j=i    ^^ 


=  \(t)   E  j  p.  .(s,t) 
j=i    "-^ 


(3.2.5) 


Defining 


m. (s,t)  =   E  j  p. .(s,t), 
j=i     ^■^ 


we  rewrite  (3.2.5),  obtaining 

a 


^  ^""i 


[m. (s,t)]  =  \(t)  m. (s,t), 


Solving, 


in.(s,t)  =  i  exp  {  J   X(u)du}, 


(3.2.6) 


(3.2.7) 


(3.2.8) 


using  the  initial  condition 

m.  (s,s)  =  i.  (3.2.9) 

The  function  m.(s,t)  shall  be  called  the  mean  function  of  the  growth 
process   [X(t)],  since  obviously 

m^(s,t)  =  EfX(t)/X(s)  =1}.  (3.2.10) 


49 


From  (3.1.1)  we  know  that 


m  (0,t)  =  n(l+a)(l+Qp  )"  , 
n 


(3.2.11) 


where  n  is  the  number  of  xonits  at  time  zero.   Equating  (3.2.11)  and 
(3.2.8)  (for  s  =  0)  implies  that,  for  the  logistic  growth  process, 


where 


X(t)  =  a0(a  +  p"*)"^, 


P  =  -  log  p . 
We  now  state 


(3.2.12) 


(3.2.13) 


Theorem  3.2. 1: 


If  fx(t) ;  t  e  I„   ]  is  a  logistic  growth  process  whose  proba- 

0  ,  00 

bility  structure  is  (3.2.1),  then 


p.   .  .,  (S,t)  31  < 


{'ti'X 


t  <i-%,t>   '*  J  ^■'o,« 


otherwise , 


where 


Proof: 


T^  ^  =  (1  +  op*)(l  +  op^)'^. 


(3.2.14) 


(3.2.15) 


The  proof  is  a  special  case  of  that  given  by  Kendall  [19]  for 
the  general  birth-death  process.   We  let  X(t)  be  defined  by  (3.2.12)  and 
p,(t)  ,  the  death  rate,  be  zero  to  obtain  the  density  given  in  (3.2.14). 
The  rigorous  proof  of  this  is  given  for  the  general  case  in  many  places, 
and  so  will  not  be  reproduced  here. 
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3. 3  Equispaced  Observation 

Assume  that  N  observations  are  made  on  the  logistic  growth 

process  [X(t) ;  t  e  I   ^}  at  equispaced  intervals  of  length  A.   Without 

losing  any  generality  in  the  results  to  be  presented,  we  henceforth 

assume  A  =  1.   Thus  [X(j);  j  e  J   }  will  be  observed,  with  X(0)  =  n 

known. 

Writing  the  set  of  observations  as  fx  ;  j  e  J    ],  the  likeli- 

j        1,N-^ 

hood  function  for  this  set  is 

L[Qf,p;  (x  ;  j  e  Jj^  j^)]  = 

=  Pf(X^  =   ^j'    ^    ^  -^l  N^/°''P-"3.  (3.3.1) 

Using  the  fact  that  [X(t)]  is  a  Markov  process  and  applying  Theorem 
3.2.1, 


.[o-.p;  (X  •  j  e  J   )]  =  TT  p[x(j)  =  x./x(j-l)  =  x. 

J        '      i=l  ^  J"-"- 


N   .  x.-l  .   X .  ,  x.-x. 

=  TT  f  J   )t  ^-^     (l-T     )  J  J-^ 

(3.3.2) 


where  x  =  n. 


Taking  the  logarithm  of  (3.3.2),  and  replacing  T.    .by  the 

defining  equation  (3.2.15),  one  gets 

N    x.-l      N 
log  L  =  E  (  ■^    )+   E  X   [log  (l+cvp-^) 

j=i  ^""j-r^^    j=i  ^'^ 

-  log  d+cyp^'  )]+  E  (x.-x.   )[log  a(p^~^-p^) 

j-1  ^  ^ 

-  log  d+Q'p'^"-^)],  (3.3.3) 
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writing  L  =  L[a,p;(x.;  j  e  J    )]  for  simplicity.   Differentiation  of 

(3.3.3)  with  respect  to  a  yields 

N-1 
^a?i-ii=  n[p(l.ap)-'  -c""]  +  Z  x.[pJ^\l+c.pJ^S-' 

-p   (l+ap   )   +^jTti^  -P    (1+ap   )  ];   (3.3.4) 
and  differentiation  with  respect  to  p  gives 
^ —  =  n[(l-p)   +  ad+Q-p)   ] 

+  S  x.[(j  +  l)ap-^  d+ap-^"^  )   -  p~ 
J-1  ^ 

-  (j-l)cyp^~^  d+ap^'^)"^] 

+  Xj^[(N-l)p'^- (1-p)"^  -  (N-l)ap^"^  d+op^"^)"^]. 

(3.3.5) 

Before  considering  (3.3.4)  and  (3.3.5)  at  greater  length,  let 

(Y. (t) ;  t  e  I„   ,  i  e  J^   ]  be  n  independent  logistic  processes,  where 
1  0,00       l,n-^ 

Y. (0)  =  1  for  each  i,  and  each  of  the  n  processes  shares  the  same  param- 
eters as  [X(t)},  (ck',p).   Defining,  for   s  <  t, 

00  00 

Hy^(s,t;    z,w)    =      Z        E     P[Y.(s)    =   ra;    Y.  (t)    =   kjz^"",  (3.3.6) 

m=l  k=m 

one  has 

00  eo 

HY^(s,t;    z,w)    =      S  PfY^(s)    =   m/Y.  (0)    =    1}   w"     Z     p[y.  (t)=  k/Y.  (s)  =  mjz'^ 
ni=l  k=m 

00  00 

TT     n-        /in-        xin-l     m      „    /k-1.    ra      ,_  .k-m     k 

=      \    ^0,s^^-^0,s^  ^        ^    (m-l)T(l-T        )  z 

nt=l  k=m 

=   T  6(w,zrT        )[i-(i-T        )5(w,z-,T      J1~^,  (3.3.7) 

vJjfa  SjU  S|t  S.U 


where 
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6(w,2;t   .  )  =  t    wz[1-z(1-t  J]~^.  (3.3.8) 

s ,  t      s ,  t  s ,  t 


Similarly,  defining 


a         00 

k  m 


H^(s,t;z,w)  =z   E   Z  P{x(s)  =  m;  X(t)  =  k/X(0)  =  n}z  w  ,  (3.3.9) 
itt=n  k=m 


one  has 


H_.(s,t;z,w)  =   E  (   ,)  T    (1-T   ) 
if  n-1   O.s    0,s 

nfcn 


_  ,k-l.   m   .,      .k-m  k 

m-l   s , t    s , t 
k=m 


=  f-^o.s  ^(-.-;%,t>[i-(i-^s.t)^^^'^=^s,t^"'^"' 


(3.3.10) 


which  implies  that 


H^(s,t;z,w)  =  [Hy.  (s,t;z,w)]   =   IT  Hy.  (s  ,  t ;  z  ,  w) .         (3.3.11) 

i=l 


Hence 


n 
X(t)  =   E  Y.(t)  (t  e  I„   ).      (3.3.12) 

i=l   ^  0'° 

If  we  rewrite  (3.3.3)  for  the  random  sequence 

{X(j)  =  X^;  j  €  Jj^  j^},  we  get 

N 
logL(Q',p;X^ X^)  =  n  cp^(Q/,p)  +   E  X.  cp.(a,p),      (3.3.13) 

where 
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-1    -1  ^      '^i-'^ 
:  log  [(l+Qp)[a(l-p)]   }  +  n    E  log  (^^   _^j     if  j  =  0 

I:  1=1         1-1 


9.(a,p)  =  ^   log  {(l+Q-p   )  [pd+ap   )]   } 


,   N-1,,   -,,    N-l-l^ 

log  [o-p      (l-pjd+G'P     )    j 


if  j  =  N. 

(3.3.14) 


But  in  view  of  (3.3.12),  this  can  be  written 

log  Ua.p;   Y^.Y^,...  ,Y^)  = 

N   n 
=  ncp„(ck',p)  +  E   E  Y.  .  ^).(.Q:,p) 
°       j=l  1=1  ^''     J 


=  E  Z.  , 
i=l  " 


(3.3.15) 


where 


Y.  =  (Y.  ,,Y.  „,...  ,Y.  J, 
—1      1,1   1,2       1 ,N 


(3.3.16) 


and 


Y.  .  =  Y.(j), 


Z.  =  9n(cv,p)  +   E  cp.(Q/,p)Y.  . 


It  follows  from  the  discussion  leading  up  to  (3.3.12)  that 


(3.3.17) 


2.  =  9_(q',o)  +  E  9.(0-, p)y.  . 
1    0        .^^     J      i,j 


log  pCl-  .a.p)  . 


(3.3.18) 


where 


p(j^.,a,p)    =  P[Y.  .^  =  yi,l'---  -^i.N  =  ^i.N^  ^"-^-^^^ 
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We  are  now  ready  to  state  and  prove 


Theorem  3.3.1:   The  equations 


a  log  L(ff,p;  \,\, X^) 


ba 


d  log  L(a,p;  x^,..  .  ,X^) 
dp 


=  0 


(3.3.20) 


have  a  solution  (a,p)    which  is  weakly  consistent  for  (o-ip).   Fvirthermore, 
the  asymptotic  distribution  of  [/v^i   (S^Q')  ;  vOi   (p-p)]  as  n  -♦  <»  is 
normal,  with  null  mean  vector  and  variance-covariance  matrix  E,  where 


-1 


11       12 

a      a 


12       22 

a      a 


(3.3.21) 


with 


11    2,,     -2    -2 

a   =  p  (1+ck'p)   -  a 

^v^  ^1   ^^■.    J,-l  r  2(j  +  l),,    J  +  ls-2   2(j-l)      j-l)~, 
+  E  (l+a)(l+Q'p  )   [p      (l+ap   )   -p      (l+Q-p     ] 


/-.   ^/■,    N.-l^  -2   2(N-1),,    N-1--2, 
+  (l+Q')(l+ck'P  )  [a     -p  (l+op   )   ], 


(3.3.22) 


12  -2 

a   =  -  (l+ap) 


N-1 


+   E  (l+C^)(l+Qp^)"^[(j-l)p^'^(l+Qp^"^)"^-(j  +  l)p^(l+Qp^'^^)'^] 


^■.   ^/■.    N  -1^.^  _  2(N-1),^    N-1,-2^ 
-  (1+Q')(l+Qp  )   [(N-l)p       (l+op    )   ], 


(3.3.23) 
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and 

22    2-2-2 
a  =   0!   (l+op)    -  (1-p) 

+  E  (l+Ck')(l+ap^)"^  [-p~^  +  [(j  +  l)a^p^'^-j(j  +  l)cyp'^"^] 

X  (l+ap-^^S"^+  [(j-l)(j-2)cypJ"^-(j-l)a2p^^J-2^ 

X  (1+Q'p"^"^)"^}+  (l+cv)(l+C^p^)~^  [[(N-l)(N-2)ap^"^ 

-  (N-l)a^p^^^"^]  (l+c'P^~S"^+(l-p)"^  +  (N-l)p"^]  .    (3.3.24) 


Proof: 

Write  p  =  p(y  ;  a.p),  9  =  a,    and  9„  =  p  for  convenience. 
Chanda  [5]  has  shown  that  the  following  three  conditions  are  suffi- 
cient for  the  theorem  to  be  true: 

(i)   For  all  (e^^.Gg)  en=  [(0,»)  x  (0,1)], 

3 

3  log  p  9  log  p    ,  B  log  p 

ae.     '  89  Se    '  ^^^  dg  59  de    ^'^^^*  ^°^  ^^"^°^*  ^^^  y- 

1      1  J         i  j   k 

and  for  all  i,j,k  e  J 

(ii)   Defining  F(-)  to  be  a  summable  function  when 

E  F(y^)  <  CO,  (3.3.25) 

^i 

where  E  is  the  summation  over  almost  all  y,  we  must 

have,  for  all   (9, ,9_)  e  n,   |||-f  <  F. (y) , 
12         o9      1  ~ 

1 

1  J  1  J   k 

where  F. (y)  and  F.  .  (y)  are  summable  functions,  and 
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^± 

M  being  a  finite  positive  constant.   This  condition 
must  be  satisfied  for  i,j,k  e  J    . 

(iii)   For  all  (6,,92^  ^^'  *^®  matrix  J  =  KJ^  .(Q    ,Q^))\ 
where 

jr^      1       J 
is  positive  definite  and  (j|  is  finite. 


(i)   Condition  (i)  will  be  established  as  soon  as 

ir— — ,  TT-r— =— — ,  and  ^^  .s-  -A   are  shov,Ti  to  exist  for  r  e  J^  „,  and 
Se.  '  36. d9.      de.oe.dO,  0,N 

1    1  J        1  J   k 

i,j,k  €  J    .   This  follows  from  the  form  of  log  p(y.;Q',p),  which 

from  (3.3. 18)  is 


log  p(y.  ;cv,p)=  E  <p(a,p)y.     .,  (3.3.26) 

where  we  define  y.  ^  -    1. 
1,0 

But  by  inspection  of  (3.3.14),  one  sees  that  cp  (a,p)  is  the 
logarithm  of  a  continuous  finite  function  for  all  (cv,p)  e  Q.      Thus 
it  is  easily  shown  that  derivatives  up  to  third  order  exist  in  Q. 

(ii)   It  follows  from  (3.3.26)  that 

N 
p(^.;Q',p)  =  exp  [  E  cp.(Q',p)y.  .}.  (3.3.27) 

1  j=0  "^  ^'"^ 
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Thus 


SP. 


N       3cp       N 

S  y-    -^   exp  {  E  cp.  y.  .} 


N 

2  y. 


a?. 


1  ,r  o9.   ~i   1   2 
r=0        J 


(3.3.28) 


which  is  bounded  by  the  summable  function 


F.(y.)  =  [  Z 


J  ~i 


r=0 


(3.3.29) 


for  from  (i)  the  terms  -^^     are  finite,  and  Efy.   }  < 

J 
Similarly, 


i\ 


ae.se. 
1    J 


iar     ^    (ae:;^k,rP(^k=^'V 


N     a  Cp 


r=0        1     J 


N     N     acp      acp 

+     E       E     (■        '  '         ' 
r=0   s=0 


VaerJ  (arJ^k.r  ^k.s  P^-^k'^i-V 


,     (3.3.30) 


which  requires  (i)  and  finite  second  order  moment  of  y   in  order  to  be 

bounded  by  the  summable  function 

,2 


N 
^■^      ^  r^O 


a  cp 


ae.ae. 
1    J 


^    ^k.NP^-^k'^'V 


N        N 
+     E       E 
r=0   s=0 


a:p  Sep 

^r  ^  s 

ae.  ae~ 

1  J 


y        p(y   ;e  ,e„).  (3.3.31) 

■*^k,N  ^  ~k'    1'    2^ 


Finally, 
a   log  p^ 


ae. ae.ae 

1      J      k 


N      a  cp 

r 

^„  ae. ae.ae    ^s.r 

r=0        1      J      k 


(3.3.32) 
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is  bounded  by 

(3.3.33) 


"ij.k^^s^  =  ^s.N 


r=0 


i  J   k 

which  has  finite  expected  value  by  (i)  and  the  finite  expected  value  of 
y    .   Thus  (ii)  is  established. 

(iii)   Chanda  has  shown  that  J  will  be  positive  definite  as 
long  as  p  is  a  density  function.   That  |j|  is  finite  follows  from  the 
fact  that  (  Iq^  ^)  (   36^  ^)  ^^  finite  (from  (i))  for  all  (9,9)  e  Q. 

Thus  the  proof  of  Theorem  3.3.1  is  complete.   A  closed  form  has 
not  been  found  for  the  solution  to  (3.3.20).   Some  numerical  results 
using  an  iterative  procedure  will  be  presented  in  Chapter  IV.   Also,  we 
attempted  to  establish  stronger  consistency  properties  for  (a.p)  ,    using 
results  of  Billingsley  [2]  and  Wald  [27],  but  have  had  no  success  as  yet. 

3. 4   Continuous  Observation  in  [0,T] 

Assume  that  the  logistic  growth  process  fX(t) ;  t  e  I^   ]  is 

0 ,  °^ 

observed  continuously  in  [0,T]  =  I    .   That  is,  [X(t) ;  t  e  I    ]  will 
be  observed,  with  X(0)  =  n  known. 

The  problem  of  estimating  a   and  p  will  be  approached  as  follows: 
first,  divide  the  interval  I     into  N  equal  intervals  of  infinitesimal 
length  h.   Assume  that  observations  will  be  made  only  at  the  end  of 
each  of  the  N  intervals;  that  is,  [X(jh) ;  j  e  J^  ^}  will  be  observed, 
where  Nh  =  T.   From  (3.3.2)  we  can  write  the  likelihood  function  for 
such  an  observation: 

L[a,p;  (Xj^;  j  e  Jq  jj)] 


N   ,  x.^-1   ,  X  .  X..  -X. 

->  (J-I)h  - 

(j-l)h,jh^        ^    (j-l)h,jh^ 


=  ^  (x "'''"  -iV^-  ^^^  ..)"^^"'^Ni-T,..  ,..  S'""  "^^-'^^ 


j-1    (j-l)h 

(3.4.1) 
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where   x^  =  "   and   T^  ^  is  defined  in  (3.2.15).   Note  that  to  obtain 

the  continuous  observation  in  I    ,  one  must  let  h  -•  0   and  N  -*  » 

such  that  Nh  =  T.   We  thus  want  to  consider  the  properties  of 

log  [lCq-.p;  (X  •  J  e  Jq  N^]}  as  h  -  0. 

We  first  define  the  concept  of  "limit  in  mean"  (l.i.m. ). 

If  a  sequence  of  random  variables   fz  ,Z  ,...,Z  , . . . ]  are  such  that 

12       m    ■' 

lim  E[(Z^  -  Z)^}  =  0,  ,    (3.4.2) 

m  -*  °= 

then  fz  }  is  said  to  approach  Z  as  a  limit  in  the  mean  square  sense,  or 

simply  as  a  limit  in  mean.   We  will  write 

l.i.m.  Z^  =  Z  .  (3.4.3) 

m  -»  00 

We  now  state 

Theorem  3.4. 1: 
The 


limit  in  mean  of  log  fL[a,p;(X.^;  j  e  J   ^ J  ] ]  as 

Jh  0,N      -' 

"h   =   T)    does   not    exist.      However,    for   fixed    (a    ,p    )    e  Q, 

l.i.m.    [logLEcp;    (X.^;    j    ^  Jo,N^^-^°SL[c.o,Pq;(X.^;    JeJ^^^)]} 

T 
=  /      (log  §      -   log   §°)    dX(t) 
0  ^  ^ 

T 
+  /     X(t)[§^    -   §°]    dt,  (3.4.4) 


Nh^T 

T 


where  we  define 


§^  =   Oi&p    n+ap    )        ,  (3.4.5) 

^t  =   ^O^oPo^^+^oPo^'^     •  (3.4.6) 


60 


The  stochastic  integrals  on  the  right-hand  side  of  (3.4.4)  are  defined 
in  the  usual  sense,  and  remember  3  =  -  log  p. 

Before  proving  Theorem  3.4.1,  we  will  need  the  following  lemmas. 


Lemma  3. 4. 1: 


lim   ,  N  -J      T 


lim   J.  N 

J 
The 

lim   ,  N 


h  -  0  -^  Z   log  (1  -  T  )f 

does  not  exist,  but  for  any  fixed  ia   ,p    )    e  Q 


^'■'"    r  ^  r    '  0  -il  I 

!■*!  if,  '  '°^L^'  -  ^(J-i)h,jh><^  -  \j-i)h,jh>    J/ 

T 

=  /   (log  §^  -  log  §^)dt,  (3.4.8) 


lim  ,  N 
1  -  0  I  E 
Nh  =  T  jzrl 

T 

r 

0 

writing  (s  <  t) 


%.t=  (1-  Vo^^^- Vo>''-  ^"-"-"^ 


Proof  of  Lemma  3.4.1: 


Using  (3.2.15) , 


r-,  j^^  f^  (j-l)h  -1 

(j-l)h,jh 

=  (1  +  Q-p^^)  [1  +  ap^^exp  (0h)]"^ 

=  fl  +  (1  +  ap^^)"^[ap^^(exp[Ph}  -1)]}-1 

=  [1  +  h§.^  +  0(h^)]"^.  (3.4.10) 

jh 
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Hence 


^°^  ^J-l)h,jh  =  -  ^°^  [i.h^.^.och^)] 

=  -  h  §.^  +  O(h^),  (3.4.11) 

Jh 

from  which  (3.4.7)  follows  immediately. 

But  from  (3.4. 10) 

1  -  ''f,    i^h  ih  =  f^  §.,  +0(h^)][l  +  h  §   +0(h^)]'^,      (3.4.12) 
(j-l)h,jh       Jh  Jh 

lira 

which  implies  that  h  -*  0  flog  (1-T..^.,   .^)]  does  not  exist.   Now 
Nh  =  T  (J-I)h,jh 

(j-l)h,jh       (j-l)h,jh 

=  [h§  .^  +  0(h^)] [1 +  h§°  +  0(h^)] [h§°  +  0(h^)]"^ 
jn  jh  jh 

X  [1  +  h§jj^  +  0(h^)]"^ 

=  §jh^^jh^'^  ^^  ^   ^^'"^^  •  (3.4.13) 

We  substitute  (3.4.13)  into  the  left-hand  side  of  (3.4.8)  to  complete 
the  proof. 


Lemma  3. 4. 2  : 

If  we  define,  for   s  <  t  S  u, 

R  (t,u)  =  C  (t,u)  -  m^  (s,t)my  (s,u)  (3.4.14) 

s         s         -^s      •'^s 

where 

C  (t,u)  =:  Efx(u)  X(t)/(X(s)  =  X  )]  (3.4.15) 

s  s  -' 

and  mx  (s,t)  is  defined  in  (3.2.10),  then 

R^(t,u)  =  (^(p^-p  )(l+ck'p^)  [(1+Q'p")(  l+O'p  *)]""'■  .  (3.4.16) 
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Proof  of  Lenma  3.4.2: 

Using  (3.2. 26)  , 

Cg(t,u)  =  E[X(t)  [E[X(u)/X(t)  =  x^]]/(X(s)  =  x^)3 

=    (1  +   ap*)  (1  +    Q'p")""''    C    (t,t).  (3.4.17) 

And  from  Theorem  3.2.1, 

E{[X(t)    -   x^][X(t)    -   x^-l]/(X(s)    =    x^)} 
/X  +j-l 


J=0  s 


=    (x  +l)x    (1-T        )^(T      J~^.  (3.4.18) 

s  s  s  ,  t  s  ,  t 


Thus 


C     (t,t)     ^     (X    +1)X     (1-T        j'^iT        J~^ 

s  s  s  s ,  t  s ,  t 

+    (2x  +l)EfX(t)/(X(s)    =   X   )}    -   X    (x  +1) 

s  ■•  s   ■'  s     s 

=   X    (T      ^)'^(x  +1-T      J.  (3.4.19) 

s      s  ,  t  s  s  ,  t 

Substitution  of  (3.4.19)  into  (3.4.17)  gives 

C  (t,u)  =  x  (T    T    )~^    (x+l-T  J.  (3.4.20) 

s         ss.ts.u      s     s,t 

Then  using  (3.4.20)  in  (3.4.14)  yields 

R  (t,u)  =  x  (T    T  J~'^    (1-T  J,  .       (3.4.21) 

s         ss,us,t         s,t 

which  is  (3.4.16)  upon  substitution  of  (3.2.15)  in  (3.4.21). 
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Lemma  3.4.3: 

The  stochastic  integral 

l.i.m.  N 
I,  =  h  -  0  Z     X(jh)  §.^  h  (3.4.22) 

exists  in  the  mean  square  sense  for  all  (a,p)  e  Q.   We  shall  write  this 
limiting  random  variable 

T 

I   =  /  X(t)  §   dt.  (3.4.23) 

Proof  of  Lenna  3.4.3: 

Using  a  result  from  Cramer  and  Leadbetter  [7] ,  sufficient  condi- 
tions for  the  existence  of  I   are  that  R  (t,u)  be  of  bounded  variation 

in  [Iq.T  ^  -^O.T^  ^""^  *^^^ 

T   T 
J^  =  /  /  §^  §^  RQ(t,u)  dt  du  (3.4.24) 


T   T 

r .' 

0  0 

exist. 

Since,  from  (3.4.16), 

0  S  R^(t,u)  <  Xq  Q'd  +  or),  (3.4.25) 

the  first  of  these  conditions  is  satisfied.   And  since  ^^  is  finite  for 

^t 

any  (Q',p)  e  H  and  all   *  ^  ^n  t'  *^^  integral  (3.4.24)  exists.   Thus 
the  lemma  is  proved. 

Lemma  3.4.4: 


The  stochastic  integral 


l.i.m.   N 


I,  =  h  -  0    E   (log  §.,)tX(jh)  -  X((j-l)h)]  (3,4.26) 

^   Nh  =  T   j  =  l       J^ 

exists  in  the  mean  square  sense  for  all  (Q',p)  e  Q. 
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We  shall  -A-rite  this  limiting  random  variable 

T 
I-  =  r   (log  §^)  dX(t).  (3.4.27) 

^   0        * 

Proof  of  Lemma  3.4.4: 

Again  from  Cramer  and  Leadbetter  [7] ,  sufficient  conditions  for 
the  existence  of  I   are  that  R  (t,u)  be  of  bounded  variation  and  that 


X     1 

J^=f     /   (log  §^)(log  5^)  d^^^[R^(t,u)]  (3.4.28) 


T   T 

r ; 

0   0 

exist.   The  first  of  these  has  already  been  shown,  and  by  differentia- 
tion of  R  (t,u),  one  has 


T  T 
Jg  =  /  /  (log  §^)(log  §^)[a3(l+a)]  p 


0   0 

[(l+Q-p*)  d+Q-p")]"^  dt  du  .  (3.4.29) 

The  integrand  is  finite  in  Q  and  [l     X  I    },  which  completes  the 
proof. 

Proof  of  Theorem  3.4.1: 

From  equation  (3.4.1) 

log  L[o,p;  (X.^;   j    €  j^^^)] 

=     H     (       ^  .)+  E  X,.  ^.^  log  [T,.  ...     .J 

3  =  1   ^(j-l)h-^^   j  =  l   ^J-^>^       (J-I)h,jh 

^  j/V-^J-l)h^  ^°^f^-\j-l)hjh^-  ^^•^•^«> 

The  third  term  on  the  right-hand  side  of  (3.4.30)  has  no  limit  in  the 
mean  square  sense  because  of  the  first  part  of  Lemma  3.4.1.   We  thus 
consider 
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log  L[a,p;(Xj^;j  e  J^^^^)]  -  log  LqEq-^.p^;  (X^^^;  j  s  J^^^)] 

=  jfl  ^a-l)h  ^'°^  ^j-l)h  Jh  -  ^°^  \j-l)h,jh> 

N  0-1 

+   E   (X.^  -X,.  ,.^)  log  [(1-T  .,)(1-T  )    ] 

.^j^   jh     (j-l)h    ^       (j-l)h,jh      (j-l)h,jh 


(3.4.31) 
for  any  ((!■„, p^)  e  Q. 

Using  lemmas  3.4.3  and  3.4.4,  we  have 

log  L*  fa,p;cyQ,pQ;  (X(t);  t  e  I^  ,j,)} 

T  T 

=  r  X(t)(§°  -§.)  dt+  r  (log  §  -log  §^  dX(t),    (3.4.32) 
0         ^  ^       0 

writing 

log  L*  [a,p;aQ,pQ;  (X(t);  t  e  I^  ,^)3 

1.  i.m. 
=  h  -0  (log  lEcv.p;  (X  ;  j  e  J^  ^)] 
Nh  =T  -^ 

This  completes  the  proof  of  the  theorem.   The  idea  of  resolving  the 
limit  problem  by  considering  the  ratio  of  likelihood  functions,  as  in 
(3.4.31),  is  from  Bartlett  [1]. 

By  using  the  expression  (3.3.12),  we  may  rewrite  (3.4.32) 
to  obtain 

*  " 

log  L   [a,p;cy^,p^;  [X(t)  ;  t  e  I^  ^] }  =   E  Z^  .,         (3.4.34) 

j  =  l 
where 

T  T 

Z    =/  Y  (t)(r-§  )dt+/  (log  §  -log  r)dY  (t).         (3.4.35) 
'0  0 
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fZ   . ;  i  €  J    ]  is  a  sequence  of  independent,  identically  distributed 
random  variables.   We  will  be  interested  in 


and 


^^2£^=  E  {/  [a-^-p*<l.„p')-^dY^(t) 

1=1    0 

T 
-3/  p^d  +  ap*)"^  Y.(t)  dtj  (3.4.36) 

*     n     T 

^  ^11^  =  ^  {!  p"\i  +  ap*-3t)(i  +  apS"^dY.(t) 

T 
-0'/  p*(l  +  Q'p*-0t)(l  +  ap*)~^Y.  (t)  dtj-  .    (3.4.37) 


T 

r 

0 

Since  L   is  not  the  sum  of  probability  densities,  as  is  the 
usual  case  for  likelihood  functions,  we  will  need  the  following 

Lemma  3.4.5: 

>      *  * 

„ro  log  L     „rB  log  L  , 
Ef— ^^  =  ^i         S3    ^  =  °-  (3.4.38) 

Proof: 

From  (3.4.36) 

*     n   ,  T 
i=l   0 
T 

r 

0 

T 


^f^-^^^=   ^  {/   [c.-^-p'(l-c.p*)-']dm  (0,t) 
i=l   0  ^   ^ 

T 
-3/  p*(l  +  ap'^)~^m  (0,t)dt} 
0  -^ . 

=  E  {a(c^l)3  J  [a"  -p*(l+ap*)"^]p*(l+cvp*) 


-^dt 


1=1  0 

T 


-  3(CH-1)  /   p*(l+ap''^)"3dtj 


0 
=  0,  (3.4.39) 
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using  the  fact  that  (Cramer-Leadbetter  [7]) 

E{dY. (t)}  =  d^E[Y.(t)] 
1    -^     t  ^  1    -■ 

=  d^[m^(0,t)].  (3.4.40) 


Similarly, 

* 


Ef^  l°g  L 


ag ]  =  ^  JP"  /  (i+c^P  -0t)  (i+ap*)~^d  m  ro,t) 

i=l      0  ^  ■'■ 

T 

0 


-or/  p  (l+ck-p  -3t)  (l+Q-p  *)"■"•  ra^(0,t)dt| 
0  -^  ■' 

T 

{aic^D  J   (l+ap*-3t)p*  (i+ap*)"^dt 
0 

T 

-  a(cH-l)  /  (l+ap*-3t)p*(i+c^p^)"^dt} 

A  J 


T 
=   E 

i=l  ^       0 


T 

r 

0 

=  °'  (3.4.41) 

which  completes  the  proof. 

We  are  now  ready  to  state  and  prove 
Theorem  3. 4.2: 


The  equations 

a  log  L*{g,p;a^,p^;  [X(t);  t  e  I^  ^] } 


ba 


=  0 


9  log  L  [q/,P;c^q,3q;  [X(t);  t  e  Iq  ^] } 

~~    §g~~ '■ =0  (3.4.42) 

have  a  solution  (or  ,0  )  which  is  weakly  consistent  for  (a,0).  Further- 
more, the  asymptotic  distribution  of  [^  (i%)  .  ^  ^^*_^^^  as  n  -^  -  is 
normal,  with  null  mean  vector  and  variance-covariance  matrix  Z* ,    where 
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and 


* 

E  = 


a..     a. 


11 


12 


^12     ^22 


*      -2   2  2 

Now,    a.,  =  A  (o;,^  X„_  +  cu„_  X_  -  2a)._uj„_X._)  , 


'11 


12   22     22  "11      12  22"12 


"=^22^  ^'^^2  \l   ^  ^?1  ^22  -  2<^ll'^12^12^  ' 


♦       —2  2 

^12  =  -  ^   ["^12^22^11^  "^ll'^12^22-  ^"^12  ^  ^l"^22^  ^12^  ' 


with,  writing  Q     =   en     and   9 


ij  -"   n  ae.de.  J  > 


ij 


ij 


ae.ae. 
1    J 


-■M( 


9  log  L  \  /9  log  L 


se. 


ae. 


(3.4.43) 
(3.4.44) 
(3.4.45) 

(3.4.46) 
(3.4.47) 


and 


^  =  '"ll  "^22  -  ^12 


(3.4.48) 


Proof: 


Writing 

log  [p*  [a,P;cvQ,0Q;  (y.(t);  t  e  Iq^^)]} 


0 
=  J     y^(t)(5^-§^)clt  +  /  (log  §^  -  log  §^)dy.(t),      (3.4.49) 


we  have  from  (3.4.34)  that 


log  L  [a,?;a^,&^;    [x(t) ;  t  e  I^^^]] 


=  E  log  p.  [a,&;a^,&Q;    (y.(t);  t  e  I^  ^)], 


(3.4.50) 
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As  mentioned  prior  to  the  statement  of  Lenma  3.4.5,  we  cannot  regard 
{p.;  i  e  J    }  as  a  sequence  of  probability  density  functions.   However, 
as  we  shall  see,  this  is  not  essential  to  the  proof  of  the  theorem. 

In  the  proofs  given  by  Cramer  [6]  for  the  single  parameter  case 
and  Qianda  [5]  for  the   multiparameter  case,  the  only  property  of  the 
density  function  crucial  to  the  proofs  is 

Efii^]  =  0.  (3.4.51) 

where  f  is  the  density  function  and  9  some  parameter  of  f.   The  property 
holds  as  soon  as   I'T^I  is  bounded  by  an  integrable  function  (over  the 
sample  space) . 

But  in  the  present  case  we  have,  from  Lemma  3.4.5, 

J       1=1        J 
which  implies 

S  log  p* 

for  j  ^  J-i  o-   Thus  the  assumption  that   p.  is  a  probability  density, 

^'^  ap*     ^ 

and,  in  fact,  the  requirement  that   I'Sq~I   i^  integrable  (or  summable 

J 
in  this  case)  are  unnecessary. 

We  now  must  establish  the  three  conditions  given  in  the  proof  of 

Theorem  3.3.1  to  complete  the  proof  of  this  theorem. 

(i)  To  see  that  log  p.  is  dif f erentiable  up  to  the  third 

order  for  almost  all  y. ,  we  need  only  note  that  § 

and  log  §   are  dif f erentiable  to  that  order;  this  is 

easily  shown  using  (3.4.5). 
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(ii)  As  mentioned,  we  no  longer  need  the  summability  condition 

Sp.  3  p 

for  |-r^|-   The  additional  requirement  that  |    ^    |  be 
o  w .  0  8  .  o  9, 

J  J   k 

bounded  by  a  summable  function  is  used  such  that 
2  * 

3^      r         .-,  S'p. 


[f  ^3= 


and  hence 

E 


2  *  * 

rd  log  p..  -  S  log  P  *   . 

foe) = -  4(^e)  (^^sF^)}  •      *^-  ^- "' 


J      k 

But  we  shall  not  make  use  of  the  property  (3.4.55),  so  it  will  not  be 

^2  * 

o  p. 

necessary  to  have  f -,   s-  |  bounded  by  a  summable  function. 

Thus,  we  need  only  establish  the  condition  that  Nr;:— =r;r— w^ 

<H     «(y.),  where  E[H.    |i(y)]  <  M.  some  finite,  positive  constant. 

Using    (3.4.49) , 

3  *  3 

S     log  p .  T      3  § 

'sOr^'  =  '  -/   de.oe  d6,  yi^^>^* 

J      k      X  0  J      k      i 

.3 


7  r  -1  ^  ^t       -2/  ^  ^t  \  /^^t\ 


(3.4.56) 


j      k      .e  j      k' 

-2/   ^'^t    Y^^tN        ,-2/^Y^ 

^t  Vde.se,  Ase,  y     ^t  vae,  aeJvae. 

J      k  k  k      .C  J 

J       k       >a 

Since,  for  every  (6  , 9„)  e  Q,  we  can  easily  bound  §   and  its  derivatives 
1   J  t 

used  in  (3.4.56)  for  all  t  el,  we  can  find  H.    Z^-)  ^s  soon  as 

T  T 

^fj  y.(t)dt3  and  EfJ  dy.(t)}  <  «,  which  follows  from  the  proof  of 


0  0 

Lemma  3.4.5 
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(iii)  The  third  condition  required  for  Theorem  3.4.2  to  be  true 


is  that  J  -    ((J.  .(9,9))  )  is  positive  definite  and  that 
1 )  J   1   ■^ 


|J(  is  finite,  where 


-    \.  ..  (3.4.57) 

From    (3.4.36)    and    (3.4.37), 

T     T 
"^1    "  /     /   [Q-'^^d+ap*)"^]  [a'^^d+cyp"^)"^]  d        R    (t,u) 
0      0  '" 


and 


i      1 

-2    (»  r>     t+u  t  u      -2 

+   3     J  J  P         [(1+ap    )  (1+ap    )]       R„  (t,u)dt   du 

0  0 

T      T 
-20/     /      [cy"%'^(l+cyp*)"^]  p"(l+cvp")"^d^RQ(t,u)du, 


(3.4.58) 

nX       =   3"^  /    /   [a'^-p'^d+ap^)"^]  (l+ap"-3u)(l+ap")"^d       R_(t,u) 
•^0      0  t,u  u 

T      T 
+  0^  I    S  p*^"(l+ap^-3t)  [(l+ap'*')  (l+c^p")]"^R^j(t,u)dtdu 


0      0 

T      T 

r  ; 

0      0 

T      T 


-/    /  [(l+ap'^-3t)  (l+ap''^)~^p''(l+ap")' 
0      0 


+  aCa     -p    (1+cvp)  (1+ap    )      ]  p^d+ap^-pu)  d+ap")"    ] 

X  d^  RQ(t,u)du,  (3.4.59) 


T      T 
^"^22  =  ^~    I    I     (l+ap*-0t)  (l+ap"-Pu)  [(l+ap*)  (l+ap")]~^ 


'22 

0      0 


><^,u\^*'"> 


7Z 


T  T 
+  a  /  /p  """"(l+ap  -3t)  (l+ap^-Pu)  [d+ap  Xl+Q-p")] 


0   0 


X  R  (t,u)clt  du 

T  T 

-  2c3~'^  J    /  (l+Q'p*-pt)(l+ap'^-0u)(l+ap*)"^  (l+ap"")"^ 


0   0 


Xd^RQ(t,u)du.   (3.4.60) 


For  all  (c^.P)  e  f},  R  (t,u),  given  by  Lemma  3.4.2,  is  finite,  as  are  its 
derivatives.   Thus  X. .  is  the  integral  of  a  finite  integrand  over  a 
finite  interval,  and  so  is  finite.   This  completes  the  proof  of  the 
theorem. 

Thus  we  have  shown  that  (5,3)  have  the  properties  usually  asso- 
ciated with  maximum  likelihood  estimators,  losing  only  the  property  that 

^logp.      ,  ,9  log  p.  .  .a  log  p.  , -, 

<-w^}  -  -  K  -^r^)(-ae— )}•  "■^•-> 

J   k  J         k 

that  is  to  say, 

"'jk  =  -V  ^'-'-'^^ 

* 
Wien  this  property  holds,  the  form  of  E  is  greatly  simplified;  since  we 

* 
cannot  assume  (3.4.62)  here,  calculation  of  E  is  tedious.   In  addition 

to  X.  .' s   given  by  (3.4.58)  -  (3.4.60),  we  will  need 

"^11^  -  (OH-DPJ  p  [ad+Ck-p  )'*]"■'■  dt,  (3.4.63) 


J,„  =  -  (Q4-1)   r  (1+Q?p  )    [p   (l+Q-p  )    -  0tp 

^^  0 

X(l+cyp^)  ~^]dt,       (3.4.64) 

U)„„  =  -  Ck'(QH-l)P~''"  J  p*  [(H-Q'p*)"'^-3t(l+Q'p*)~^]^dt.       (3,4.65) 


and 


0 
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No  closed  form  has  been  found  for  the  solution  to  the  estimation 
equations  (3.4.36)  and  (3.4.37),  nor  could  a  more  convenient  form  be 
found  for  the  elements  of  E  .   It  is  conjectured  that  iterative  numer- 
ical methods  can  be  used,  but  we  leave  this  particular  topic  open  to 
further  research. 

3. 5  Arrival  Time  Observation 

Assume  that  instead  of  observing  the  logistic  process  [X(t) ] 
itself,  we  observe  only  the  times  at  which  the  process  increases;  that  is, 

the  "arrival  times"  are  observed.   Observation  is  terminated  when  the 

th 
time  of  the  N   increase  has  been  recorded.   Hence  we  will  observe 


fT^;  J  e  J^  j^}.  where 


x(T.)  is  < 
J 


<n+j      if   t<T. 
J 


^  n  +  j      if   t  ^  T.,  (3.5.1) 


where   x(0)  =  n. 

The  distribution  of  this  sequence  of  random  variables  is  easily- 
obtained  using  Theorem  3.2.1.   For  any   9^0, 

P  {T,   .  -  T.  :S  0/(T.  =  t.  ;  i  e  J^  .)] 

=  P[x(t.  +  G)  -X(t.)  >k/(t.;  i  e  J,  .)] 
J         J        1       l,j  ■' 

=  P[X(t.+e)  -  X(t.)  ^  k/C.},  (3.5.2) 

where 
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x(t)  -  n 
x(t)  =  n  +  1 


Cj=< 


x(t)  =  n  +  j  -  1 
'^  x(t)  =  n  +  j 


if  t  <  t 


if  t^  ^  t  <  t^ 


if  t.  ,  S  t  <  t. 

if   t  =  t.. 
J 


(3.5.3) 


Since   [X(t)}  is  Markov,  we  have 

PfT,  .  -  T.  S  e/(T.  =  t.  ;  i  e  J   .)] 


=  P[X(t.+  e)  -  X(t.)  ^  k/(X(t.)  =  n  +  j)} 
=  P{X(t.  +  e)  2  n  +  j  +  k/(X(t.)  =  n  +  j)] 


=  .\  C^.J-l)<^..t^.e'""'<'-^.t..9>' 

i=k     ^        J   J  J   J 


(3.5.4) 


Since  (3.5.4)  is  a  function  of  t.  only,  it  follows  that  [T . ;  j  ^  J    } 
is  a  Markov  sequence,  and  from  (3.5.4)  the  distribution  function  is 


given  by 


F       (e;t.)  =  p[t.    St  +e/r  =  t.} 


=  1  -  (T 


>^'J 


t  ,t .+' 
J  3 


(3.5.5) 


requiring   9^0. 

To  follow  the  procedure  suggested  by  Moran  [22]  for  several  dif- 
ferent birth  and  death  processes  would  be  to  calculate  the  likelihood 
function 


N 


L( 


c/,p;  t  ,. .  .  ,t  )  =  Tf  f     (a,p  ;  t  ,  9)  , 


where 


J     J+1 


J+1 


[F^   (9;t.)], 
J 


(3.5.6) 


(3.5.7) 
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which  exists  in  7}   since  T.   ^   .,  and  thus  F     (9;t  ),  is  absolutely 

3      3  J  +  1 

continuous  for   9  S:  0. 

However,  Moran's  approach  can  be  shown  unsatisfactory  for  this 
case  by  observing  that,  from  (3.5.5), 

lim   F     (e;t.) 
e  -  »    j+1    ^ 

t.  +  e       *•   1  n  ■ 
=  lim  [1  -  [(1+ap  ^       )(l+ap  •^)"  ]  ^^  ] 

Q-»oo 

=  1  -  (1+ap  J)-^"+J^  .  (3.5.8) 

Hence  there  is  a  non-zero  probability  that,  given  T.  =  t.,  T.    will 

not  be  finite.   Thus   EfT.  ,/(T.  =  t.)}  does  not  exist,  and  the  likeli- 
•■  J  +  1   J     J 

hood  function  given  by  (3.5.6)  is  not  useful  for  estimating  (a,p) . 

Perhaps  a  conditional  likelihood  could  be  used  to  get  estimation 
equations.   Fxirther  study  of  the  problem  is  needed. 

3.6   Prediction 

Assume  that  N   equispaced  observations  will  be  made  on  the 
logistic  growth  process  [X(t) ;  t  e  I   ^} ,  as  in  3.3.   Thus,  based  upon 
observations  taken  on  [X(j);  j  e  J    },  we  will  predict  the  value  of  the 
time  series  at  time  N  +  m,  that  is,  X(N+m) . 

Restricting  our  attention  to  the  predictor  which  minimizes  the 
mean  square  error  of  prediction,  we  define 

i^^\m)  =   E[X(N+m)/[X(j)  =  x.;  j  e  J^  ^]} 

=  EfX(N+m)/(X(N)  =  Xj^)} 

=  m  (N,N+m) 
"^ 

=   x^(Uc.p^(l.ap ''"'")"'.  (3.6.1) 
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using  (3.2.10).   Then   x    (m)  minimizes  the  mean  square  error  of  pre- 

N 

diction,    which   is 

MSE     =   E[[X(N+m)    -  X   (m)]"} 

=  E{R      (N+m,N+m)} 

^r^         ,-,  N.   .     N      N+m^  .,  N+m.    -2, 

=   E{X^  Q-Cl+ap    )(p    -p         Xl+Q-p         )        } 

^-      X     N.,      in^,^  N+m^ -2  ,^^^^ 

=  nad+Q')  p     (1-p    )(1+Q'p        )        ,  (3.6.2) 

where  X(0)    =   n. 


If   a  and  p    are  unknown,    consider   the  predictor 

-(2)  .    .  /.    -  'N,  ,,    >.  -N+m  -1 

X        (m)   =   X   (l+cyp     Xl+cyo         ) 


(3.6.3) 


-^(2) 

Denoting  the  mean  square  error  of  x   (m)  by  MSE  ,  it  follows  that 

N  ^ 

MSE„   -  MSE,    =   E[[X7/(m)    -  X^-'Cm)]^} 
2  1  N  N 

T^rJ^r,^  N.    _  N+m  -1 

=    E[X~[(l+Ck'P     )    (l+Q'p  ) 

.,    >»  aN.    ,,    A  -N+m   -1    2  t-3   c    A\ 

-   (1+ck'p    )  (1+ap        )      ]     }.  (3.6.4) 

Define 

U  =    1  +   ap^  ,  (3.6.5) 

V=    1  +   ap''^"''",  ^                               (3.6.6) 

5a  =  a  —  a,  (3.6.7) 
and 

6p  =  p"   -  p.  (3.6.8) 
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Then 


6U  =   (1  +  ap   )    -   (1  +  ap^) 


=  Ofp        -  ap 

=    (^04-0')  (5  p+p)        -    ap 

=  p    fiof  +  Nap  5  p  +   0    (n      )  ,  (3.6.9) 

making  use  of   the   fact    that    the  maximum  likelihood   estimators    are  weakly 
consistent.       Sinilarly, 

o\  =   (1+c^p         )   -   (1  +  ap        ) 

N+m   c  ,„     .        N+m-1     .  ,    -1 

=  P  00+    (N+m)Q'p  6  p  +   0    (n      ) .  (3.6.  10) 


Hence 


UV"^  =    [5u  +    (l+cyp^  ]    [6v  +    (1+cvp^^")  ]"^ 

=  (1+c.p^  (1+c.p''-''")-^  (i+cp^'^^-^eu 

/I  N-    ,,  N+m^  -2     t  ,    -1 

-d+ap    )  (l+ap        )         6v  +  0    (n     ).  (3.6.11) 

No»   we   can   get    an   approximation   for    (3.6.4)    by   taking 
MSE^   -   -MSE^   =    EfX^CUV"^    -    (l+^p^   d+cp'^^'")  "^1  ^  } 

«E(X^)    E[[Uv"^-(l+c^p^   (l+apN+'")-l]2-j 

=   Cq(N,N)(1+cvp^^'")"^    E{[6u-(l+cvp^5V]^] 

=  p^^  n[n+a(l-p^(l+a)"^](l+cy)^    [(l+cvp^  (l+ap^^"")  ]  "^ 
xf  a^^[l-(l+cvp^  (l+c.p^'^"''")'^'"]^  +   2a^2  ^P"^ 

X[N-(2N+m)p'"(l+ap^  d+op^^'")"^    +    (N+m) 

V     r.^^    r-,  N.  2  _  N+m   -2,  2    -2 

X     p        (1+ap    )     (l+ap        )      ]    +   cr       a  p 

[N   -    (N+m)(l+c^p^  (l+c.pN+'")-^    p'"]2j,  (3.6.12) 
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Perhaps  (3.6.12)  is  only  a  crude  approximation  for  (3.6.4),  but  inspec- 
tion of  MSE   (3.6.2)  and  the  approximation  for  MSE   -  MSE   (3.6.12)  leads 
X  «       1 

to  several  interesting  observations: 

N 

1.  MSE   is  proportional  to  p  ;  that  is,  MSE   decreases  exponen- 
tially as  N  increases.   This  is  intuitively  plausible,  since  the  corre- 
lation between  X(N)  and  X(N+m)  is  given  by 

Corr  [X(N) ,  X(N+m) } 

-1/2 

=  R  (N,N+m)[R  (N,N)  R  (N+m,  N+m) ] 

rf,      N.  ,,   N+m. -1,1/2  .„  „  ,„. 

=  [(1-p  )(l-p    )   ]     .  "  (3.6.13) 

Hence  the  correlation  is  a  raonotonically  increasing  function  of  N,  and 
it  follows  that  X(N)  will  contain  more  information  about  X(N+m)  when 
N  is  large;  thus  the  decrease  in  prediction  error. 

2.  MSE   is  proportional  to  n.   Thus,  contrary  to  the  error 
associated  with  the  maximum  likelihood  estimators  of  the  logistic  param- 
eters, the  error  of  prediction  increases  with  the  initial  size  of  the 
process.   This  is  perhaps  partially  explained  by  the  fact  that  the 
process  variance  at  any  time  is  proportional  to  n  (Lemma  3.4.2); 
heuristically ,  the  population  or  process  size  is  less  deterministic  for 

large  populations. 

2N 

3.  MSE   -  MSE   is  proportional  to  p   .   Hence  the  additional 

prediction  error  incurred  by  parameter  estimation  also  decreases  as 
N  increases,  and  at  a  faster  rate  than  MSE  . 

4.  MSE   -  MSE   is  proportional  to  n. 

^      1 

Observation  (4)  is  surprising  if  true;  but  it  depends  most 
directly  upon  the  accuracy  of  approximation  (3.6.12).   Thus,  though 
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this  may  be  just  another  of  the  logistic  process'  strange  characteristics, 

one  must  question  the  credibility  of  (3.6.12).   Rather,  one  expects  the 

increase  in  prediction  error  to  be  of  order  one,  following  the  general 

pattern  that  a  decrease  in  estimator  variance  produces  a  corresponding 

decrease  in  the  ratio  of  MSE„  -  MSE,  to  MSE  . 

2      11 


CHAPTER  IV 
AN  EMPIRICAL  COMPARISON 

4. 1  Introduction 

Rhodes  [25]  has  given  a  method  of  estimating  the  logistic 
parameters  which  receives  wide  application.   We  present  his  method  in 
this  chapter,  with  an  attempt  to  examine  the  estimators'  sampling  prop- 
erties.  These  shall  be  shown  to  be  analytically  intractable;  thus 
a  Monte  Carlo  experiment  is  performed  in  order  to  compare  these  esti- 
mators with  those  presented  in  the  last  chapter.   Finally,  the  logistic 
model  will  be  fitted  to  the  population  of  conterminous  United  States, 
using  both  Rhodes'  and  maximum  likelihood  estimation  procedures. 

4.2  Rhodes'  Estimators 

Recall  that  the  mean  function  for  the  logistic  growth  process, 
given  by  (3.2,12) ,  is 

E{X(t)/X(0)  =  n}  =  m  (0,t)  =  nil+a)  (1+a p^)~^  .  (3.2.12) 

Upon  inspection,  it  is  seen  that 

[m  (0,t+l)]"^=p[m  (0,t)]"^+  (l-p)[n(l+a)]"^.  (4.2.1) 

n  n 

Rhodes  thus  defines 

Y(t)  =  [X(t)]"^,  (4.2.2) 
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and,  observing  the  random  variables  (Y(j)=Y.)  Y-,Y  ,  ...,Y  ,  he  finds 
estimators  for  p  and  y  ii^ 

ytH-i  =  p^  -^  y  *  '^.N-i'  ^^'^'^^ 

where 

Y  =  (l-p)[n(l+cv)]"^,  (4.2.4) 

and  y   is  the  observed  value  of  Y  . 

The  estimators,  obtained  by  minimizing 
N-1 

s  =  r  [y^^^  -  (py^  +  Y)l  .  (4.2.5) 

1=1 


Pr  =  L^f^  <^t-yi)(^.ry2>JL^f/yt-^>  J 


and 


where 


Yr  =  ^2  -  °r  ^1  ' 


and 


(4.2.6) 


(4.2.7) 


-1  N-^ 
y,  =  (N-l)    E  y.  (4.2.8) 

■^         t=l  ^ 


_1  N-l 
y„  =  (N-l)     E  y.  ,  .  (4.2.9) 

^  t=l   *+^ 

Using  (4.2.4),  one  gets  an  estimate  for  ck': 

a^  =    (l-p)(nY^)"-'-  -1  .  (4.2.10) 

Rhodes  then  shows  by  application  that  one  obtains  reasonably 
close  agreement  between  projection  and  reality,  using  these  estimated 
values.   To  my  knowledge  no  probabilistic  study  of  the  properties  of 
these  estimators  has  been  made. 


82 


4. 3   Sampling  Properties  of  Rhodes'  Estimators 
Define,  for  0  S  s  S  t   and  u  >  0, 

3  (s,t)  =  E{Y(t)/Y(s)  =  n"^},  (4.3.1) 

§^(s,t)  =  E[Y^(t)/Y(s)  =  n"^3,  (4.3.2) 

and 

Tl  (s,t,t+u)  =  EfY(t+u)Y(t)/Y(s)  =  n"-^}.  (4.3.3) 

Now  fix   s,t  and  u,  and  write  B  ,§   and  71  for  (4.3.1),  (4.3.2)  and 

n     n  n 

(4.3.3),    respectively.      We  now  prove 

Theorem  4.3.1: 

n-1 
(i)    g     =    v"  log    T   -      E     j         v"  ^;  (4.3.4) 

j  =  l 

(ii)    §     ^    ^   {-^  [3[cos"^(l-T)}^   -  6tt  cos~^(1-t)   +   2n  ] 

°"^        -1  -1  -^"^      -1     -k 

+      E      [j         log   T   -  j  Z     k        V      ]];  (4.3.5) 

j=l  k=l 

and 

(iii)    T|      ,   =    v[Tl     -  n"^   3*],  (4.3.6) 

n+1  n  n 


writing 


and  defining 


T=    (1+ap*)  (1+cyp^)"^,  (4.3.7) 

P*  =    3    (s,t+u),  (4.3.8) 

n  n 


V  =   T(T-l)"-'".  (4.3.9) 
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Proof: 


It  follows  from  Theorem  3.2.1  that 


PfY(t)  =  j"Vy(s)  =  n~^]  =  P[X(t)  =  j/X(s)  =  n] 


o 


T   (1-T) 


J-n 


=  < 


if  j  =  n,n+l, . 


otherwise.         (4.3,10) 


Thus 


But 


J=n 


J+n-1 
=  Z     (j+n)"" 

j=0         ^  n-1 


T^l-T)-^  . 


J=n+1 


1     "  -1       /^■'"'^"■'•N 

=  n   V  Z  [n(j+n)    -1]  (    ,  )  t"(1-t)-^ 


=  V(3   -  n"^), 
n 


Notice  that 

0^  =  E[Y(t)/Y(s)  =  1] 


(4.3.11) 


(4.3.12) 


=   S  j"^  T(l-T)^~^ 

=  V  log  T  . 


(4.3.13) 
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Now,  defining  the  generating  function 


Po(z)  =   E  0  z  ,  (4.3.14) 

n=l 

we  have,  using  (4.3.12)  and  (4.3.13), 

OB 

P.(z)  =  B  z  +  vz  P-(z)  -  vz   Z  n""^  z'^  .  (4.3.15) 

P  ^  P  __, 

Rearranging, 

Pg(z)   =    (1   -   vz)'      vz[log   T+   log    (l~z)],  (4.3.16) 

which  implies  that 

n-1  _^ 

B     =  v"  log  T  -  E  j~   v°  •^,  (4.3.1) 

as  was  to  be  shown. 

(ii)  Similarly, 

(j+n)    ( 
J=0 


"        2  /'■^■^"■'•N   n      i 
§^  =   E  (j+n)    (  n_i  j  "^  (l-*^)  .  (4.3.17) 


and 


5... = .  vj-'"  rrV""<-'^" 


=  n"^  V  E  (j+n)"^[n(j+n)"^  -1]  t'^(I-t)^ 
j=0 

=  V  [?   -  n"^  3  ].  (4.3.18) 

n        n 
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Noticing  that,  from  a  result  in  Davis  [8], 
§^  =  E{Y^(t)/Y(s)  =1] 

00 

J  =  l 

=  ^  V  f3[cos"-'-(l-T)]^  -  6n  cos"'^  (1-T)  +  2n^},        (4.3.19) 

we  define  the  generating  function 


Pp(z)  =   E  §   z  .  (4.3.20) 

n=l 

It  then  follows  from  (4.3.18)  that 


-1 


But 


00      <» 

-1 


(4.3.21) 


P  (z)  ^  §  z  +  V  z  [P_(z)  -En   P  z  ]  , 
^  *      n=l 

which  implies  that 

n-1 
5  =  ^^"   §^  +  Z     3         v"""-^  p  (4.3.22) 

j=l  '' 

Upon  substitution  of  (4.3.19)  and  (4.3.1),  this  is  (4.3.2). 

(iii)  From  (4.3.3) 

00      00 

71  =   E   E  [(j+n)(k+j+n)]"^ 
"^       j=0  k=0 

X  PfX(t+u)    =    k+j+n/X(t)    =   j+n}P[X(t) 

=  j+n/X(s)    =   n]    .  (4.3.23) 


\^3^  =      E       E     [(j+n)(k+j  +  n)] 
j=l  k=0 

X  P{X(t+u)    =   k+j+n/X(t)    =   j+n]P[X(t)    =   j+n/X(s)    =   n+1] 
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00       00 

=  n~  V  E    E  (k+j  +  n)~   [l+n(j+n)~  ] 
j=0   k=0 

X  PfX(t+u)  =  k+j+n/X(t)  =  j+n}P[X(t)  =  j+n/X(s)  =  n] 
=  V  [7]^  -  n"^  p*],  (4.3.24) 

as  was  to  be  shown.   This  completes  the  proof  of  Theorem  4.3.1. 

The  reason  generating  functions  were  not  employed  to  further 
resolve  (4.3.24)  is  that  no  closed  form  could  be  found  for  T)  . 

It  is  not  difficult  to  use  Theorem  4.3.1  to  obtain  approximations 
for  the  first  order  sampling  properties  of  Rhodes'  estimators.   Tlie 
expressions  are  tedious,  the  computing  process  was  extremely  time  con- 
suming (and  hence  expensive) ,  and  the  few  numerical  results  obtained 
were  nonsensical.   This  is  probably  due  to  rounding  errors  resulting 
from  the  great  number  of  calculations  involved,  or  perhaps  from  the 
inaccuracy  of  approximations  made  in  order  to  obtain  these  expected 
values. 

Second  order  properties  of  Rhodes'  estimators  would  involve 
fourth  order  moments  of  {Y(t)],  which  are  intractable.   However,  one 
can  perform  Monte  Carlo  experiments  to  at  least  compare  the  estimators 
for  specific  processes. 

4. 4  Monte  Carlo  Experiments 

A  computer  program  was  written  for  the  IBM  360  which 

(1)  simulates  a  logistic  growth  process  given  n,  a,    p    and  N; 

(2)  uses  this  sequence  to  calculate  the  maximum  likelihood 
estimates  given  in  3.3,  using  the  Newton-Rhapson  iterative 
procedure; 
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(3)  calculates  Z,    the  asjmptotic  variance-covariance  matrix 
for  these  estimators; 

(4)  calculates  Rhodes'  estimates  for  a  and  p ,  using  the  method 
given  in  4.2. 

The  results  of  several  of  the  experiments  are  presented.   The  format  is 
as  follows:   first,  a  table  is  given  which  lists  the  generated  logistic 
sequence  and  the  sequence  of  expected  values.   This  table  is  followed 
by  a  graph  depicting  both  the  generated  sequence  and  the  expected 
sequence.   Finally,  a  second  table  gives  the  maximum  likelihood  esti- 
mates and  the  corresponding  asymptotic  variance-covariance  matrix, 
using  the  first  third,  first  two-thirds,  and  all  of  the  generated 
sequence,  respectively.   Rhodes'  estimators  are  also  calculated  for 
each  case. 

Three  such  experiments  are  presented,  each  with  a  different 
value  of  n  and  a.      We  have  conducted  several  other  experiments,  and 
those  presented  here  are  exemplary.   Changes  in  p  do  not  significantly 
affect  the  results,  except  that  rounding  errors  in  the  computations 
affected  the  maximum  likelihood  estimates  for  p  (p    ^  -) .      These  cases 
are  practically  uninteresting,  though,  since  for  small  p  the  process 
quickly  approaches  an  asymptotic  value,  a  characteristic  uncommon  to 
most  economic  and  other  real  sequences. 

Note  that  Rhodes'  estimates  are  unreliable  when  N  is  small, 
whereas  the  effect  of  N  on  the  maximum  likelihood  estimates  is  negli- 
gible.  In  addition,  we  could  obtain  asymptotic  confidence  intervals 
for  the  parameters,  using  the  maximum  likelihood  estimates  and  E.   We 
also  note  that  the  bias  in  ^  seems  to  depend  primarily  on  the  relationship 


Table  4. 1 

Generated  and  Expected  Sequence  for 
n  =  40 ,  a  =    10 . 0 ,  and  p  =  0 . 8 
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m  (0,i) 


1 

55 

59.46 

2 

58 

71.90 

3 

72 

86,34 

4 

84 

102.88 

5 

94 

121.50 

6 

110 

142.07 

7 

128 

164.32 

8 

144 

187 .  86 

9 

164 

212.18 

10 

176 

236.69 

11 

192 

260.79 

12 

211 

283.92 

13 

225 

305.60 

14 

244 

325.48 

15 

262 

343.35 

i 

X. 

1 

m    (0 
n 

,i) 

16 

267 

359, 

,13 

17 

276 

372, 

.84 

18 

291 

384, 

.58 

19 

296 

394. 

.52 

20 

302 

402, 

,84 

21 

307 

409. 

76 

22 

309 

415. 

48 

23 

313 

420. 

16 

24 

314 

423. 

98 

25 

318 

427. 

09 

26 

319 

429. 

61 

27 

321 

431. 

65 

28 

321 

433. 

30 

29 

324 

434. 

62 

30 

326 

435. 

68 

89 


•    •    • 


400- 


350- 


300- 

X(t) 

250- 


o  o 


o  o 


o  o 


o  o 


o   o 


200- 


150- 


100- 


50 

n  =  40ei_ 


•        o 
o 


•   o 
•   o 

o 


10 


o  -  observed  value 
•  -  expected  value 


15 


~So 


25 


— I 
30 


Fig.  4. 1.   Generated  and  Expected  Sequence  for 
n  =  40 ,  a  =  10 .  0 ,  and  p  =  0  .  S 
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Table  4.2 

Maximum  Likelihood  Estimates,    E  and  Rhodes' 

Estimates   for   Data    in  Table  4. 1   for 

n  =   40,      a  =    10.0    and    p  =   0.8 


a     =      5.909 
p     =      0.794 


a     =   69.464 
r 

0     =      0.873 


N  =    10 


E  = 


81.242  0.603 


0.603  0.00493 


N  =   20 


or     = 
P    = 


7.134 

0.810 


9.747 
0.851 


Z  = 


2.801  -0.0170 


-0.0170  0.  182  xlO 


-3 


N  =   30 


a    = 

7.045 

P        = 

0.808 

A 

^r  = 

A 

D_    = 

8.290 
0.845 

E  = 


3.247  -0.0162 


-0.0162  0.112  xlO 


-3 
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Table  4.3 

Generated  and  Expected  Sequence  for 
n  =  100,  Q-  =  5.0,  and  p  =  0.8 


i 

X. 

1 

m  (0,i) 
n 

i 

X. 

1 

m  (0,i) 
n 

1 

146 

142.86 

16 

645 

539.28 

2 

179 

168.54 

17 

655 

550.42 

3 

225 

196.85 

18 

668 

559.67 

4 

265 

227.41 

19 

677 

567.30 

5 

304 

259.66 

20 

682 

573.55 

6 

339 

292.89 

21 

686 

578.65 

7 

386 

326.29 

22 

691 

582.80 

8 

425 

359.05 

23 

696 

586. 16 

9 

472 

390.40 

24 

697 

588.88 

10 

509 

419.73 

25 

698 

591.07 

11 

534 

446.56 

26 

700 

592.83 

12 

566 

470.63 

27 

702 

594.25 

13 

591 

491.84 

28 

702 

595.39 

14 

607 

510.24 

29 

702 

596.31 

15 

629 

525.98 

30 

703 

597.04 

92 


7001 

650 

600 

550 

500 

450- 
X(t) 

400-1 

350- 
300- 


ooooooooo 


•   •  • 


0  •   •  o  e 


250 


200 


o  • 


o  -observed  value 
•  -expected  value 


150- 


n=IOoiL 


Fig.  4.2.   Generated  and  Expected  Sequence  for 
n  =  100,  cy  =  5.0,  and  p  =  0.8 
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Table  4.4 

Maximum  Likelihood  Estimates,  Y,   and  Rhodes' 
Estimates  for  Data  in  Table  4.3  for 
n  =  100,  Q/  =  5.0  and  p  =  0.8 


N  =  10 


a    = 

4.523 

P     = 

0.752 

^r  = 

5.222 

0.749 

z  = 


1.169 


0.0221 


0.0221  0.769  XlO 


-3 


N  =    20 


a    - 

5.086 

P        = 

0.772 

A 

5.729 

A 

P,.   = 

0.759 

r  = 


0.308  -0.354  xlO 


-2    -1 


-0.354X10"^      0.101  XIO"^ 


N  =    30 


a    = 

5.057 

A 

P        = 

0.774 

A 

5.873 

D_   = 

0.762 

z  = 


0.323  -0.326  XlO 


-2 


-0.326  XlO    ^      0.647  X  lo"'* 


Table  4.5 

Generated  and  Expected  Sequence  for 
n  =  400,  Q-  =  3.0,  and  p  =  0.8 
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i 

X. 

1 

m    (0,i) 
n 

i 

X. 

1 

m    (0,i) 
n 

1 

553 

547.95 

16 

1439 

1498.75 

2 

626 

630.92 

17 

1448 

1517.96 

3 

689 

717.88 

18 

1468 

1533.69 

4 

767 

806.84 

19 

1478 

1546.51 

5 

849 

895.64 

20 

1487 

1556.92 

6 

942 

982. 11 

21 

1497 

1566.35 

7 

1023 

1064.31 

22 

1502 

1572. 16 

8 

1108 

1140.70 

23 

1508 

1577.65 

9 

1172 

1210. 18 

24 

1515 

1582.07 

10 

1226 

1272. 17 

25 

1522 

1585.62 

11 

1276 

1326.53 

26 

1522 

1588.48 

12 

1323 

1373.48 

27 

1524 

1590.77 

13 

1361 

1413.50 

28 

1527 

1592.61 

14 

1400 

1447. 14 

29 

1529 

1594.08 

15 

1421 

1475.41 

30 

1529 

1595.26 

95 


16001 


1500 


o    o  o    o    ° 


o   o  o 


»400 


o 


1300 


•   o 


1200 


1100 
X(t) 

lOOOj 


90O 


800 


700- 


600- 


o -observed  value 
•  -expected  value 


50O 


n=400tiL. 


10 


15 


20 


— I — 
25 


Fig,  4.3.   Generated  and  Expected  Sequence  for 
n  =  400,  a  =   3.0,  and  p  =  0.8 


30 


96 


Table   4.6 

Maximum  Likelihood   Estimates,    Z  and  Rhodes' 
Estimates   for   Data   in  Table  4.5   for 
n  =   400,    a  =   3.0    and  p    =   0. 8 


N  =    10 


a    = 

2 

470 

P       = 

0 

768 

^r  = 

3 

472 

A 

0 

837 

E  = 


-2   -1 


0.0663  0.206  X 10 


0.206  XIO   ^      0.195  XlO    ^ 


N  =   20 


a    = 

2.576 

p"    = 

0.784 

2.962 

D_   = 

0.822 

s  = 


-3   -, 


0.0306  -0.539  X 10 


-0.539  XIO"        0.359  X  lo" 


N  =   30 


a    = 

2.578 

p"    = 

0.790 

^  = 

2.901 

Pr    = 

0.820 

E  = 


-3 


0.0315  -0.512  XIO 


-0.512  XIO    ^      0.242  X  10~^ 
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of  the  generated  to  the  expected  sequence.   That  is,  when  the  generated 
sequence  falls  short  of  the  expected  sequence,  a   tends  to  underestimate  a. 

We  conclude  by  fitting  the  logistic  model  to  the  population  of 
conterminous  United  States.   We  use  the  census  results  at  each  decade 
from  1800  to  1960  as  the  data  to  which  the  model  is  fitted,  treating 
each  decade  as  one  unit  of  time.   Using  the  population  given  by  the 
1800  census  as   X(0)  =  n,  we  calculate  the  maximum  likelihood  and  Rhodes' 
estimates  for  a   and  p  for  two  sequences:   that  made  up  of  the  populations 
from  1810  to  1910  and  that  of  those  from  1810  to  i960.   We  then  use  the 
estimates  based  upon  the  latter  sequence  to  predict  the  population  of 
conterminous  United  States  in  1970  by  three  methods: 

(1)  that  using  (3.6.3  )  and  the  maximum  likelihood  estimates, 

(2)  that  using  (3.6.3  )  and  Rhodes'  estimates,  and 

(3)  that  using  the  recursive  relation  (4.2.3)  and  Rhodes' 
estimates. 

Table  4.7  summarizes  the  results  of  this  study.   The  data  are 
given,  and  then  the  maximum  likelihood  estimates,  Rhodes'  estimates,  and 
an  estimate  of  E,  the  asymptotic  variance-covariance  matrix  for  (cy.p)  , 
are  presented  for  the  two  aforementioned  sequences.   We  complete  the 
table  by  giving  the  predictions  for  the  1970  population  in  the  order 
presented  above,  and  finally  the  data  are  given  for  convenience. 

Note  the  rather  significant  discrepancy  in  the  estimates  for  the 
two  sequences,  illustrating  a  point  made  by  Granger  [14]:  the  logistic 
model  is  rarely  useful  in  making  long-range  predictions.   However,  as 
shown  by  the  prediction  of  the  1970  population,  the  model  has  applica- 
tion when  one  confines  extrapolations  to  shorter  time  spans. 


(a)  Data 


Table  4.7 


Summary  of  Results  of  Study 
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Year 

Population 

(millions) 

1800 

5.308 

1810 

7.240 

1820 

9.638 

1830 

12.866 

1840 

17.069 

1850 

23.192 

1860 

31.443 

1870 

38.558 

1880 

50.156 

Year 

Population 

(millions) 

1890 

62.948 

1900 

75.995 

1910 

91.972 

1920 

105.711 

1930 

122.775 

1940 

131.669 

1950 

150.697 

1960 

178.464 

1970 

202.113 

(b)   Maximum  Likelihood  Estimates,  E  and  Rhodes'  Estimates 


Data  I  (1810  -  1910) 


a     =  36.882 
p*  =   0.733 


a  =  48.085 

r 

d"  =   0.740 


a     =  41.767 
p   =   0.778 


S  = 


0.459  X  10 


-2 


0.  134  X  10 


-4  -1 


0.134  X  10 


-4 


0.363  X  10 


-7 


Data  II  (1810  -  1960) 


E  = 


-3 


0,821  X  10    -0.150  X  10 


-5  -1 


-0.150  X  10 


-5 


0.308  X  10 


-8 


a     =  43.955 

r 

D   =   0.739 
(c)   Predictions  of  1970  Conterminous  Population 
True  Value:   202.113  Millions 


Method 


Maximum  likelihood 
estimates  and  (3.6.3  ) 

Rhodes'  estimates 
and  (3.6.3  ) 

Rhodes'  estimates 
and  (4.2.3) 


Prediction  (millions) 


197.268 


191.172 


189.766 
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The  extra  effort  required  to  calculate  the  maximum  likelihood 
estimates  seems  to  be  worthwhile.   One  is  encouraged  to  try  to  apply 
the  methods  used  for  the  logistic  process  to  other  evolutive  models 
whose  parameters  are  currently  estimated  without  reference  to  probabil- 
ity structure.   Further  research  into  this  area  is  contemplated. 
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